I implemented the simplest form of XOR encryption. The XOR operation with the clrea text and the key resulst in non-ascii characters, that is an issue when sending the encrypted text. I chose to convert the unsigned char string into a HEXADECIMAL character string, only 0-9 A-F characters there and they play nice. For concersion I used the sprinf function, and the output seems to be correct.
However when decrypt the HEXADECIMAL string, I need to convert the two byte Hexadecimal "characters back into unsigned char values, and that is when I fail to find a solution, the sscanf function keep generating General protection failure at the very first instance of the conversion (I, j=0). I tried various formatting parameters "0x", "02X", "02hhx", all fail. I am out of ideas.
I looked up the formatting string and either I miss something or the datatype are wrong but it compiles with no error.
/*XOR encrypts message with the key (an MD5 hash) and returns the pointer to the allocated encrypted hexadecimal string*/
unsigned char * XorEncode (const char * message, char * key) {
int i=0,j=0;
size_t messagelen=0;
unsigned char *hexstringbuffer;
messagelen = strlen(message);
if(messagelen<=1|| strlen(key)<1) return NULL;//Error in parameters
hexstringbuffer=malloc(((messagelen*2)+1));
memset(hexstringbuffer,'\0',2*messagelen+2);
//convert the encrypted buffer to hexadecimal string
for(i = 0,j=0; i < messagelen; i++,j+=2) {
sprintf(&hexstringbuffer[j],"%02x", message[i] ^ key[i % (sizeof(key)/sizeof(char))]);// works well with or without sprinf conversion
}
hexstringbuffer[j-2]='\0'; //for j incremented before exiing the loop
return hexstringbuffer; //user must free the encrypted buffer, do not want to store it when out of scope
}
//decryptes the message of a hex stringn eeds the same key as the excription
//need to convert back the hexadecimal (ascii) array to unsigned char array
// fgor this reason I need separate decode function;
//all works if the hexadecimal conversion is omitted
unsigned char * XorDecode(const char * encryptedmessage, char * key) {
int i,j;
size_t messagelen=0;
unsigned int bytebuffer='0';
messagelen=strlen(encryptedmessage);
char *decrypted=malloc (sizeof(char)*(messagelen/2)+1);
for(i=0,j=0; i < messagelen; i++,j+=2) {
sscanf(encryptedmessage[j],"%02x",&bytebuffer); // I GET A GENERAL PROTECTION FAULT HERE
decrypted[i]=bytebuffer ^ key[i % (sizeof(key) / sizeof(char))];
}
decrypted[i]='\0';
return decrypted; //the user must free the decrypted buffer
}