Golang for infosec — Building an EDR #Part2 — YARA rules
In case you missed the first part of this serie, it was to illustrate how to use Go to read in the memory of Windows processes. With the help of Win32 API implementations we now have this valuable content. All that remains is to analyze it. If you want to go back to the previous part, it’s this way.
If you are interested in the finalized project i made, I invite you to have a look at my IRMA (Incident Response — Minimal Analysis) source code and release.
For now, let’s talk about YARA: Originally developed by Victor Alvarez of Virustotal and mainly used in malware research and detection, YARA has been done for identifying malware (or other files) by creating rules that look for certain characteristics. It was developed with the idea to describe patterns that identify particular strains or entire families of malware. It’s exactly what we need to start building an EDR in Golang.
Using YARA with Go is not so difficult. Hilko Bergen as done an incredible job with his go-yara package. So what’s the problem? Here, we don’t want YARA to be installed on the computer. Instead we will use libyara and the functions of this library to compile rules and search for match. That’s why we will need to import C code and handle some compilation tricks.
Go, C, … cgo!
As the official Golang documentation says, cgo enables the creation of Go packages that call C code. To use cgo, write normal Go code that imports a pseudo-package “C”. The Go code can then refer to types such as C.size_t, variables such as C.stdout, or functions such as C.putchar.
If the import of “C” is immediately preceded by a comment, that comment, called the preamble, is used as a header when compiling the C parts of the package. For example:
// #include <stdio.h>
// #include <errno.h>
import “C”Why should you use cgo?
I can see you coming! “C is a very powerful language and thanks to cgo we will be able to take advantage of this performance in a Go programme.” No, no and … no! First of all, Go is a language powerful enough to meet most of your needs, including the most challenging ones. In very rare cases, if you need to go back to C, then do it from beginning to end with this language. And if you want to use Go anyway, you would prefer using cgo for interfacing with your own or existing libraries, possibly with small C wrapper functions that can reduce the number of calls you need to make from Go. This is exactly what Hilko Bergen does in the go-yara package.
Why not mixing C and Go?
As James Henstridge said in a very relevant post on stackoverflow: There is fairly high overhead in calling C/C++ code via cgo. So in general, you are best off trying to minimise the number of CGo calls you make. This is even more true for complex programmes requiring the use of goroutine. There are a number of aspects of how the Go runtime sets up this sort of threads that can break the expectations of many pieces of C code:
- Goroutines run on a relatively small stack, handling stack growth through segmented stacks (old versions) or by copying (new versions).
- Threads created by the Go runtime may not interact properly with libpthread’s thread local storage implementation.
- Go reuses OS threads to run multiple Goroutines. If the C code called a blocking system call or otherwise monopolised the thread, it could be detrimental to other goroutines.
libyara compilation and cgo on Windows
cgo is not very easy to use under Windows, in particular because it does not know how to recognize the libraries in *.lib format when they are compiled with Microsoft Visual C++.
In order to mitigate this problem, it will be necessary for you:
- to install mingw (I suggest to do it via msys2)
- to compile YARA from GCC
- to use pkg-config to allow yara to be linked to your programs
It may seem complex to you, and I admit that it was difficult for me the first time because I went from disappointment to disappointment. But once you know the procedure, it’s not so hard. I invite you to consult the documentation i’ve made to install my IRMA project. It’s deliberatly step by step documented in order to get there.
Compile your YARA rules
Now you’ve got mingw install, libyara correctly compiled and that you set-up your environmet variables and PATH to help the C linker, you’re ready to go. Start by installing go-yara:
go get github.com/hillu/go-yaraand then install this library with:
go install github.com/hillu/go-yaraIf it worked without any error, cgo is well configured and compilation has been done successfully. If not, go back to my windows compilation readme
Now you are ready to use libyara from go-yara package, you’ll see that it will provide everything from yara rules compilation to analyze. Let’s start by compiling our rules in order to be able to use them later.
// LoadYaraRules compile yara rules from specified paths and return a pointer to the yara compiler
func LoadYaraRules(path []string) (compiler *yara.Compiler, err error) {
compiler, err = yara.NewCompiler() if err != nil {
return nil, errors.New("Failed to initialize YARA compiler")
} for _, dir := range path {
f, err := os.Open(dir)
if err != nil{
log.Println("[ERROR]", "Could not open rule file ", dir, err)
} namespace := filepath.Base(dir)[:len(filepath.Base(dir))-4] if err = compiler.AddFile(f, namespace); err != nil{
log.Println("[ERROR]", "Could not load rule file ", dir, err)
} f.Close()
}return compiler, nil}
Search for YARA match
From your compiled rules, it’s time to search for match:
// PerformYaraScan use provided YARA rules and search for match in the given byte slicefunc PerformYaraScan(data []byte, rules *yara.Rules) yara.MatchRules {
result, err := YaraScan(data, rules) if err != nil{
log.Println("[ERROR]", err)
}return result
}
And that’s it, from there you can retrieve the content of the memory of your processes, then scan it with your YARA rules and process each match as you see fit.This will not be enough to make a proper EDR and we will need to work on detecting many more tactics, techniques and procedures used by malwares, but this will be the subject of the next section.
