Search for a command to run...
An article about one of golang's most powerful features
I am a Python/Go developer mostly working on developing REST, GRPC APIs and various different backend services. Along with developing them , I usually put them in a container and deploy them to the cloud.
No comments yet. Be the first to comment.
Concurrency is neat. If you use it responsibly.
And why you should use them less than you think.
Caching API responses with the database developers love to use
A really nice module indeed!
What are interfaces, and how do they work
Concurrency and parallelism. Quite a complex topic. Various programming languages tackle this issue in different ways. Some are really complicated, some a bit less. But, I would dare to say that Go really does well in this area. The reason why it handles concurrency and parallelism so well, is because it was built with in the 21st century, when multi-core processors are an industry standard, and speed of execution is of the essence.
What is a goroutine?
A Goroutine is essentially a very lightweight substitute for a thread. If you are coming from Java, you will probably know that a single Java thread allocates 1MB of memory by default. On the other hand, a singe goroutine allocates only 2kb (!) . It can dynamically add more memory, but it will not waste it.
How to implement a goroutine
Now let's look at some code. Let's say I wan't to access a third party API to get some information. I need quite a bit of info, and I need it fast. Knowing that most of the time it takes to retrieve data from a simple HTTP request is spent waiting for the remote server to respond, I decide to use concurrency.
To make things simple , I will go step by step and first do it the synchronous way. First I need a function that sends an API call and returns the request body:
func getData(url string) string {
r, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
return string(body)
}
As we know the request , being a network call, can go wrong so I check for any errors.
I also call defer r.Body.Close() because that way we tell this function to close the request body after it is finished with it, same way we would close a file after reading it's content.
Finally, we assign a body variable the value returned from a call to the ioutil.ReadAll function, which takes a parameter that implements the io.Reader interface ( one that is implemented quite often in Go ) , and returns a byte array, which we can easily convert to a string afterwards.
Now let's call this function in our main function. I will use my favorite API , the Rick and Morty API, which I use in all of my examples ( luckily, not very many people implement the code from my articles, or this API server would be down 24/7) :
func main() {
r := getData("https://rickandmortyapi.com/api/character/100")
fmt.Println(r)
}
So great, we got some info on character 100, who's name is very fortunate - "Bubonic Plague" . Nice. But, we actually need info on all of the Rick and Morty characters , not just Mr.Plague. And we don't want to wait tens or hundreds of seconds do get it, we need it now! Goroutines to the rescue :
func getDataFaster(url string, c chan string) {
r, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
c <- string(body)
}
Whoa! That's not the syntax you expected , right? I mean , wtf is this arrow doing? And what is a chan?
In order for me to explain let me show you the version you expected, and how we would call it in the main function ( spoiler alert : it will not work as expected ) :
func main() {
for i := 1; i < 200; i++ {
go getDataFaster("https://rickandmortyapi.com/api/character/" + strconv.Itoa(i))
}
}
func getDataFaster(url string) {
r, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(body))
}
For now we modified the getDataFaster function only to print the result, and not return it. I'll explain why later. We also use a for loop to get all 200 characters ( there's actually more but we really don't want to spam it so much) . Since i is an integer, we will use the strconv built in module to convert it to a string.
Now, here's the cool part. We use the go keyword to tell the go compiler that we want this function to run asynchronously, meaning run so fast it seems all 200 function calls are running simultaneously. Unfortunately, when we run this, we find that it does nothing.
Well, now is about the time to leave a F U in the comments below.
Just kidding.
The reason this doesn't return anything is because the main function is essentially a goroutine of it's own, and it just went through the for loop and exited. It doesn't care that the other 200 goroutines haven't finished their job. It finished it's job and now wants to have a beer.
Enter channels!
Remember that bit of ugly code with the arrow and chan. Those are channels. A channel is essentially a place to store some value from our goroutine and enable another goroutine (our main function in this case) to get that value. That arrow at the end basically says - instead of returning the string(body) like you would in a regular function, jam it in a channel because this is no regular function, it's asynchronous.
Ok, but how does the main function then access these values you ask? Let's see in the next code snippet:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"strconv"
)
func main() {
ch := make(chan string)
var data []string
for i := 1; i < 200; i++ {
go getDataFaster("https://rickandmortyapi.com/api/character/"+strconv.Itoa(i), ch)
data = append(data, <-ch)
}
fmt.Println(data)
}
func getDataFaster(url string, c chan string) {
r, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
c <- string(body)
}
Btw, this code includes all of the package and import declarations for folks who are just here for the copy/paste-ing .
As you can see, the first thing we do is declare a channel by using the built-in make function. A channel also needs to be statically typed , so we declare a channel that will hold only strings. Next, we declare a slice of strings called data where we plan to store all this info on all these wacky Rick and Morty characters. We enter our for loop as before, but now we have this weird arrow again. This time the channel isn't the one that is on the receiving side , but rather seems to be the one that is sending . What we're doing here is appending the value that the channel will hold for us to our data slice.
This way we are initiating a blocking operation. This means that the main function will block until a value is actually gotten from the channel. This is how we make the main function wait , instead of carelessly exiting like it did the first time.
This is also the reason why we can't assign a value to a call to a goroutine ( I said I would explain later) , because it is asynchronous, and it will not return any value, it can only add it to a channel.
Finally, we print everything to the console, and since it's fetching 200 results , your console will probably blow up like mine:
So there it is. A very elegant way of retrieving a lot of data very quickly by using goroutines and channels.
Hope you enjoyed. Thanks for reading!