Search for a command to run...
Caching API responses with the database developers love to use
I am a Python/Go developer mostly working on developing REST, GRPC APIs and various different backend services. Along with developing them , I usually put them in a container and deploy them to the cloud.
No comments yet. Be the first to comment.
Concurrency is neat. If you use it responsibly.
And why you should use them less than you think.
A really nice module indeed!
What are interfaces, and how do they work
If there's one technology that is on par with Golang on the "coolness" scale, it is definitely Redis. Backend developers love it. For four years in a row it is officially the most loved db, according to Stack Overflow’s Annual Developer Survey .
Although it is officially a NoSQL database, you'll usually use it as a cache, or maybe a pub-sub system. Some developers are advocating using it as a primary database, but I think for now that's a bit too extreme.
In this article, we'll look at some code that basically implements a couple of simple API endpoints, that use Redis to access/store the data.
I'll also be using commands like go mod init and go get to initiate a go module and get the third party package I will use to connect to my Redis instance.
By the way, I'll be running Redis in Docker, so if you have Docker installed on your local machine you should run docker run -d -p 6379:6379 redis to get it up and running. If not, either install Redis locally, or install Docker locally and then run the command above.
So, let's initate a go module and fetch the go Redis driver. I'll do this by typing the following commands in my terminal, in the directory where I'm going to write my code:
go mod init github.com/pavledjuric/go_redis_webapp
go get github.com/go-redis/redis/v8
By the way, just because I name my module with the github.com prefix, doesn't mean it's actually going to be on GH, but if I were to push this code to a git repo, then I would push it to that one.
Now I have a module and the redis driver, so let me create a simple web server:
package main
import (
"log"
"net/http"
)
func main() {
port := ":8080"
http.HandleFunc("/users", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hellooo users!!"))
})
log.Fatal(http.ListenAndServe(port, nil))
}
Boring! A simple http server with a single endpoint that doesn't do anything smart. So, let's make that handler take some data from Redis, and be able to write some data to Redis if it's a POST request. I'll also store my handlers in a separate package called handlers just to make it seem more like a real app.
Just so we're on the same page, and there's no issue with imports, your main.go file should go in a directory called cmd . That is go jargon for command, and that's where the executable usually goes. Your handlers directory and all of it's files should go in a directory called pkg. You guessed it , that's short for packages. This is how your project directory tree should look like:
Now let's write the code for our handlers.go file:
package handlers
import (
"context"
"io/ioutil"
"net/http"
"github.com/go-redis/redis/v8"
)
func UsersHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "GET":
var ctx = context.Background()
red := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
res, err := red.Get(ctx, "user").Result()
if err != nil {
w.Write([]byte("Error!"))
}
w.Write([]byte(res))
case "POST":
body, err := ioutil.ReadAll(r.Body)
if err != nil {
w.Write([]byte("Error!"))
}
defer r.Body.Close()
var ctx = context.Background()
red := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
_, err = red.SetNX("user", body, 60*time.Second).Result()
if err != nil {
w.Write([]byte("Error!"))
}
w.Write([]byte(body))
}
}
It may look a bit confusing but it's actually pretty self explanatory. We're using a switch statement that will execute code depending on the HTTP method of the request we're handling (GET or POST) .
If it's GET, we'll create a context and a connection to our Redis server. Next , we'll try to get the data that is stored in the user key in the our Redis db. If we can't find that key, that will return an error, and our server will let us know it's an error.
If the request method is POST, we'll read from the request body and store that data in the user key in Redis with a 60 second time to live. This means that after 60 seconds the data will be deleted from Redis, thus mimicking how a real caching system would work. Of course, in a real application, we would store the data permanently in an SQL or NoSQL database apart from storing it temporarily in the cache.
There's one thing I'd still like to do here. This code doesn't seem too DRY (don't repeat yourself) , since I'm instantiating the redis.NewClient twice with all the connection parameters unnecessarily. Let's extract that to a function, and create another package to store it in. I'll call the package redisconn, so now my project tree looks like this:
The code for the redis connection will go in the redisconn package. Basically, it's just a wrapper to the redis driver we downloaded, but perhaps we wan't to add some custom logic after, so it's a good idea to have it in a separate package:
package redisconn
import (
"github.com/go-redis/redis"
)
func GetRedisConnection() *redis.Client {
return redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
}
Now back to the handlers package, let's import redisconn and update the code. I also wan't to add an http status code of 404 if the data is not found, and 500 if for some reason the server crashes while reading data from the POST request (I forgot to do that initially, and I'm too lazy to update it honestly) :
package handlers
import (
"io/ioutil"
"net/http"
"github.com/pavledjuric/go_redis_webapp/pkg/redisconn"
)
func UsersHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "GET":
red := redisconn.GetRedisConnection()
res, err := red.Get("user").Result()
if err != nil {
http.Error(w, "Error!", 404)
return
}
w.Write([]byte(res))
case "POST":
body, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, "Internal server error", 500)
return
}
defer r.Body.Close()
red := redisconn.GetRedisConnection()
_, err = red.SetNX("user", body, 60*time.Second).Result()
if err != nil {
http.Error(w, "Internal server error", 500)
return
}
w.Write([]byte(body))
}
}
Ok, that definitely looks a bit cleaner now.
So let's try it out:
I first try the GET request, knowing that my Redis db is empty:
Looking good. Now I'll add my user to Redis by sending a POST request. I'll use Postman , but you can use curl or whatever you preffer:
Ok, it seems to have worked, so let's call the GET /users endpoint again:
Beautiful!
However, I'm a bit paranoid by nature, so I wan't to make sure this data is actually coming from Redis, and not some alien hackers from another planet. Let me go inside the docker container that is running Redis and check. I enter the container by running the following command
docker exec -it a1b redis-cli
This a1b is the first 3 characters of my container ID , yours will most definitely be different . You can check it by running docker ps .
Ok, so I'm in my Redis container, inside the redis cli tool. I run keys * to see that user is indeed there. I run get user to find that the data is definitely coming from Redis:
I'm truly relieved it was not the aliens.
Remeber how we added a 60 second time to live for the data inside Redis, so let's check how much time it has left by using the ttl command:
Now let's wait out those 52 seconds and check again:
Looking good. What about our API server, does he agree?
He does indeed! Seems like everything is working as intended.
In conclusion I would say that connecting Redis to your Go API server is pretty simple as you can see, and it will boost your response times significantly. Typically, you would use a SQL database as a primary db, and Redis would be in front of it. When a user would request data from your API server, the server would first check Redis for the data. If Redis has it, it will return it in a blazing fast manner. If it this data is not in Redis, the server would then retrieve the data from the SQL server (MySQL, Postgres or whatever you’re using), and on it's way back store that data in Redis, with a specified time to live (because your data is probably dynamic and changes often , so you do not want your cache to store it indefinitely) . This is known as the cache-aside pattern. There are other caching patterns like write-through, but I usually use cache-aside and I recommend it for most cases.
That's all for this one folks. Thanks for reading.