Golang | Project
Introduction Guide to RPC in Golang
All about golang RPC and how touse it
So after finishing my course work I realised that I have heavily used RPC for my projects and this blog illustrates why RPC’s are so useful and how to use them in golang.
Why do we need RPC?
Lets consider a scenario that we want to update a database which contains products data. The database is only accessible by a server and client sends the GET, PUT and POST methods to the server for all operations.
Now you are probably guessing it is just straightforward to perform api calls and then this situation is handled.
The problem with traditional API calls
When we scale our system and we are operating in a distributed systems setting or microservices setting traditional api calls have lesser to offer.
The main problems with them are these
- High Performance overhead: With frequent inter service communication and repeated packaging and unpackaging of json objects over HTTP leads to high latency and low throughput.
- No Bi directional streaming: A REST api only flows one way , in a larger system where you need to know the response for a call more than just a status code and don't want to read an entire json this becomes difficult.
- Massive Network Overhead: Its no wonder that with so many components the frequent api calls will increase the network overhead of the system.
Then why RPC
RPC provide a layer of abstraction on the communication complexity between services. The main strength of RPC’s are to execute a procedure on a remote machine as if it was a local function which simplifies the process of development.
The abstraction of network communication between nodes allows them to interact seamlessly and is significant in distributed systems as it enables scalability , fault tolerance and a centralized management of resources.
A practical Example
We are going to build a key value store which can only be accessed by a server and a client which requests data to be updated in this store.
What does a RPC need
Before we dive in lets discuss what an RPC function looks like.
There two main components of performing an RPC call
- An args or argument
- A reply
As you can guess an args contains all the parameters we wish to pass , imagine this like the json body you pass in the REST API calls.
The reply is the data we receive back from the server , instead of getting back every detail we can let the server know what all is required from it.
func (kvs *KeyValueStore) Put(args PutArgs, reply *bool) errorTWO important aspects
- The reply must be a pointer to work.
- All methods and variables should be exportable ie capital letters.
- All methods must follow this structure of arguments and reply.
This is the general structure of an RPC function , an argument which contains the data the client wants the server to have and the reply which the server fills up to let the server know the reply.
General Architecture Flow
This is how the flow of operations will look like with RPC.
Designing the Server
In golang the rpc package is net/rpc and that is what we are going to use.
Define the keyvalue struct
type KeyValueStore struct {
mu sync.RWMutex
store map[string]string
}Now lets define the args struct that the client will use to populate the data it wishes to send to the server for PUT requests
type PutArgs struct {
Key string
Value string
}This lets the server know which key and value the client wants to insert into the database.
Now lets define the GET struct, we just define the key which the client wants from the server.
type GetArgs struct {
Key string
}Now lets define the replies back , first lets start the PUT reply
type PutReply struct {
Success bool
Message string
}This defines whether the put was success and the message returned. Now we define the GET reply
type GetReply struct {
Value string
Found bool
Message string
}We add an additional field called found just for get to let the client know if this field even exists in the database
Now to define the PUT function for example is how this goes
func (kvs *KeyValueStore) Put(args *PutArgs, reply *PutReply) error {
kvs.mu.Lock()
defer kvs.mu.Unlock()
kvs.store[args.Key] = args.Value
reply.Success = true
reply.Message = "Value stored successfully"
return nil
}The args has the data we want to insert and we populate the reply with the values we want.
The complete server code will be this.
package main
import (
"fmt"
"log"
"net"
"net/rpc"
"sync"
)
type KeyValueStore struct {
mu sync.RWMutex
store map[string]string
}
type PutArgs struct {
Key string
Value string
}
type PutReply struct {
Success bool
Message string
}
type GetArgs struct {
Key string
}
type GetReply struct {
Value string
Found bool
Message string
}
func (kvs *KeyValueStore) Put(args *PutArgs, reply *PutReply) error {
kvs.mu.Lock()
defer kvs.mu.Unlock()
kvs.store[args.Key] = args.Value
reply.Success = true // return to client that operation was a success
reply.Message = "Value stored successfully of key " + args.Key
fmt.Println("Server completed PUT request for ", args.Key) // a simple server log
return nil
}
func (kvs *KeyValueStore) Get(args *GetArgs, reply *GetReply) error {
kvs.mu.RLock()
defer kvs.mu.RUnlock()
value, ok := kvs.store[args.Key]
if !ok {
reply.Found = false
reply.Message = "Key not found"
fmt.Println("Server did not find key ", args.Key) // if key not found return this message
return nil
}
reply.Value = value
reply.Found = true
reply.Message = "Key found " + args.Key // if key is found we return this message
fmt.Println("Server completed GET request for ", args.Key)
return nil
}
func main() {
kvs := &KeyValueStore{
store: make(map[string]string),
}
rpc.Register(kvs) // registers the server with key value store struct,
// server is identified with the name of struct
listener, err := net.Listen("tcp", ":8000") // register port 8000
if err != nil {
log.Fatal("Listen error:", err)
}
fmt.Println("Started Server on port 8000")
for {
conn, err := listener.Accept()
if err != nil {
log.Fatal("Accept error:", err)
}
go rpc.ServeConn(conn) // start the server
}
}Designing the Client
Now the client just has to call the server , this is done using the rpc.Dial method to get a client and then use the client to call the methods.
To call a remote method the syntax is this
client.Call("{Server Registered Name}.{Method Name}", args , reply)This is the general syntax to call rpc server methods. The client code is hence written like this
package main
import (
"fmt"
"log"
"net/rpc"
)
type PutArgs struct {
Key string
Value string
}
type PutReply struct {
Success bool
Message string
}
type GetArgs struct {
Key string
}
type GetReply struct {
Value string
Found bool
Message string
}
func main() {
// Use RPC Dial to connect to server
client, err := rpc.Dial("tcp", "localhost:8000")
if err != nil {
log.Fatal("Dialing:", err)
}
// The client Put operation
putArgs := PutArgs{Key: "first", Value: "first value"}
var putReply PutReply
err = client.Call("KeyValueStore.Put", &putArgs, &putReply)
if err != nil {
log.Fatal("Put error:", err)
}
fmt.Printf("Put result: Success=%v, Message=%s\n", putReply.Success, putReply.Message)
// The client Get operation
getArgs := GetArgs{Key: "first"}
var getReply GetReply
err = client.Call("KeyValueStore.Get", &getArgs, &getReply)
if err != nil {
log.Fatal("Get error:", err)
}
if getReply.Found {
fmt.Printf("Get result: Value=%s, Message=%s\n", getReply.Value, getReply.Message)
} else {
fmt.Printf("Get result: Key not found, Message=%s\n", getReply.Message)
}
// Try to get a non-existent key
getArgs = GetArgs{Key: "second"}
err = client.Call("KeyValueStore.Get", &getArgs, &getReply)
if err != nil {
log.Fatal("Get error:", err)
}
if getReply.Found {
fmt.Printf("Get result: Value=%s, Message=%s\n", getReply.Value, getReply.Message)
} else {
fmt.Printf("Get result: Key not found, Message=%s\n", getReply.Message)
}
}client calls a method and then reads the reply pointer to determine whether it was successful or not. This is useful as client does not have to go through enormous amount of data but just the required data.
Running the Code
Running the server
Run the server by the following command
go run server.go
Running the client
Run the client using the following command
go run client.go
And in the server side
If you like the blog please consider subscribing and liking the post.
