How To Identify Unknown Crypto Functions [ Reverse Engineering AMA ] – YouTube Dictation Transcript & Vocabulary

1.do it a lot hi folks and welcome to open analysis live AMA where we answer your reverse engineering questions if you like this kind of stuff go check out our patreon lots more reverse engineering content there with that let's get into it I I there was a question for me in the chat mishap asked how do you identify crypto algorithms and there's three things yeah so I I think the I think the first question is when you look at a function in the decompiler how do you know it's crypto and for that there's basically three things that I would use it's a context so you know how it's if you know how it's being used then you should get some spidey sense about it might be encryption or compression two is the function signature it's pretty characteristic usually for most decryption things can be if it's in place it's less characteristic because then it will be a buffer in length and and a key um but often it's a buffer in a length a buffer and a length and a and like input output key and and so this this helps but the most important one is constants or and arithmetic operations so when you look at it and it has a lot of shifts like there are a lot of shift operations or a lot of like a lot of arithmetic more more than usual more than you would usually see a lot of xor like if they're if they're appearing in blocks like like eight times the same operation this happens for this happens for Block ciphers but it like if you it the more I think the better recommendation is just more arithmetic and and bitwise operations than usual because for stream ciphers they're not necessarily arranged in blocks because it's not a feistel network or for compression algorithm is it's also not necessarily arranged in blocks but generally just more arithmetic more bitwise operations than usually and now like the question you wanted to answer is how do you know what it is like what the algorithm is and that is like I would say that is one of the high-end disciplines of reverse engineering is algorithm identification it there's the like on the easiest level you what what Washi said constants like there will be constants in most algorithms and just Googling them will often lead you to a reference implementation that you can then compare to the code you see and if constants don't help you then it gets more challenging because then what I honestly do is if I I either I have seen it before which is a like not a helpful answer like just look at it I mean it's obviously lz4 I've seen this check a thousand times it's already yeah I mean that that's not helping uh what you will have to do and uh or the best method I know is to reverse it a tiny little bit understand something that it does to the data and research that Google for things try to figure out if there's something that does something like this it's it's an agonizing process and once you've found it the pain will have been so great that you will memorize this I promise also maybe a sort of a brute force method that I've also tried before is uh if I know that it is some kind of compression then uh there are only a few very commonly used compressions algorithms out there so like it's not that difficult to to go to their Wikipedia page of it and just quickly glance over it and see if it looks at least something similar to it uh there's always a few candidates that you can just look for uh very quickly I have uh pretty much I have the same goes for crypto or just as chat GPT right exactly so whenever whenever I encounter a compression algorithm that I didn't know before I implemented in Refinery and I will read to you the list like in the beginning and I had just four and I now have one two three four five six seven eight nine ten eleven to thirteen fourteen fifteen sixteen right so there's odd ones too and 99 of them are Level impulsive variants and why why couldn't the internet just agree on one LZ algorithm why do there have to be 10 of them I do not know rant over I'll stop I have a I have a few tips which I think everybody probably knows but nobody mentioned it the first one is you need to figure out what the arguments are and you need to actually have a really clear understanding of the arguments and so oftentimes if I'm really stuck I'll use a debugger I'll just run the debugger until I get to the beginning and I'll copy the argument side so I know exactly what they are and I I tried a Brute Force first I mean you guys see me do that on stream like once I have the arguments I'll just try I have a massive library of all kinds of different compression decryption all that [ __ ] and uh before I even start that I'll look at the output so I'll look to see was it compressed or was it decrypted that's the first thing and then I kind of know okay well it's way bigger so it was compressed or oh no it's just decrypted so that's the first thing I'll do and I'll just try all of them blindly or anything that kind of makes sense right you can usually tell from the key material too depending on whether there's a initial stage to set up the key or not so if they have a if they have like a key setup function and then you can see what the output of that is usually that can tell you whether it's something some sort of AES or something like that that sometimes happens but I've recently come across a bunch of different malware that Implement modified versions of existing crypto and I can tell you that that is the hardest thing ever because it will be very close to something that you know but when you implement it in python or use one of your libraries it doesn't work and that to the the truth is I spent more time on shitty rats that had custom crypto algorithms than on anything any hard [ __ ] I've ever done hey yes [Laughter] like when they have there's like a Cha-Cha Cipher but it's like slightly modified I think or no they did the key separately from it anyway there's been a couple times where things have been just custom and those cases I have never found a good way other than just an infinite time sync of replicating it was it black matter yeah I knew one of those um one of those guys did it but yeah it's just a black hole of replicating it and then making sure what you replicated is true and then you know it's custom that's how you know but even even for modified stuff the general approach is not different I recently analyzed a sample that has a modified rc6 implementation where all left rotations have been replaced with right rotations and vice versa which was very annoying but I was I know the entire RC suit very well so I looked at it and I was sure that it's either either rc5 or rc6 and then I'm you have to look at the key derivation to figure out which one it is and I was certain it was rc6 but it didn't decrypt correctly so if you have I don't know any better way but if you're if you if you have a reference implementation and you can compare it with your decompiled code it will be painful but you will eventually identify the modification it's but as Sergey said it's it's painful I should this also sorry just to belabor this this is something that I care about and do a lot of too so it's just you guys really triggered me with this one but um it's also really important to note that a lot of the time doing the work I'm doing now I don't give a [ __ ] about what it's called Uh you should definitely just emulate it if you just need to do a config extractor or something it does not matter just just own up to the fact that you don't know what it is and implement it in your emulator uh the only reason why I used to have to figure these things out is because I used to write thread intelligence reports in which they need the name of the algorithm for some reason who knows why it's important and so I'd have to actually name the algorithms and doing that like I said was just the worst the worst job ever Sergey secretly hates threat intelligence I mean of course I don't I don't know what it is because it means I mean I will I will challenge you on this and I will yeah I mean you then know what kind of algorithm the actor used and there is some there is some interesting so for example there was this time when we saw um samples packed with these it was always a different cryptographic algorithm that was used and it was always some odd stream Cipher and it was in different samples but by knowing what the stream ciphers were um I could figure out that they were all finalists in The E something like in this in a stream Cipher competition so this tells you whoever um I don't think it was NSA stream or something is the uh the East stream side for and so what happened there is like there was this author threat actor somewhere who just went through the finalist section of this uh of this competition essentially and used all these algorithms and they were pretty unique and pretty rare So when you now find two samples both using one of these algorithms you know you know there's there's an overlap there and you only found that out by really identifying the ciphers knowing where they come from and knowing how they were like how they likely ended up in the sample so I will I will put out there that is sometimes useful to know what it is and how it came to be it's my just my opinion just my just my two cents I was I was just kidding rattle it's red intelligence is very important so that wraps it up big thanks to our panel of experts you can go check out their socials here thank you very much and if you guys want to see more reverse engineering content like this in-depth tutorials live streams your questions answered go check out our patreon lots more stuff like that there and stay tuned for the next question