Video Streaming with Go

A brief guide about implementing a video stream with Go

I watched my first video from the internet in 2004. It was a music video downloaded through itunes. Keyword there, “downloaded.” Since that time, the way most of us consume video content on the internet has changed. From just downloading, to simply streaming.

Whether it’s on YouTube or Netflix, there is ultimately a server at some location that is streaming the video you are watching. Initially, my perception of video streaming was that:

• It was something hard to implement.
• It required specialized protocols I was too lazy to learn.

Turns out I was wrong and putting together a streaming server is quite simple. At the same time, I’m not trying to downplay the challenges faced by the big players of video streaming.

Thomas William — https://unsplash.com/photos/person-holding-video-camera-4qGbMEZb56c

In this post, we’re going to setup a HTTP video streaming server with Go. This server will be in compliance with RFC 7233.

Architecture

Prior to writing any Go code, I’d like to explain how video streaming works with modern browsers. This is the default behavior out of the box and I won’t be writing any HTML for this post.

In my opinion, streaming is enabling clients to request specific portions of a resource instead of downloading it entirely. The client sends a HTTP request for the file with a header named Range. This header specifies which portion of the file the server should send back. The server will also tell the client which parts of the file it requested with a response header named Content-Range.

Here is a diagram to better illustrate the ideas outlined in the paragraph above:

The Go Code

To stream an MP4 video I’ll need a:

• A video store that can return portions of a video.
• An endpoint to interpret the client request and return the requested portion of the file.

I’ll start with the video store interface. This interface will have one function and it will be used to seek bits of a file. Here the definition of the VideoStreamer interface:

type VideoStreamer interface {
 // start and end are passed in bytes. 1024 would be 1kb
 Seek(key string, start, end int) ([]byte, error)
}

Next, I’ll add a new struct type named MockVideoStreamer . This type will have one field named Store and it will house video data. Here is the code for said type:

// MockVideoStreamer is a simple mock 
// implementation of the VideoStreamer interface
type MockVideoStreamer struct {
 Store map[string][]byte
}

With the struct type declared, I’ll proceed by updating the MockVideoStreamer type to implement the VideoStreamer interface. The Seek method will check if the provided video key exists within field Store and if it does, it will return the bytes requested. Here is the code of this implementation:

// Seek retrieves a video chunk from the store based on the key and byte range
func (m *MockVideoStreamer) Seek(key string, start, end int) ([]byte, error) {
 // Retrieve the video data from the store using the key

   videoData, exists := m.Store[key]
   if !exists {
      return nil, fmt.Errorf("video not found")
   }
  
   // Ensure the range is within the bounds of the video data
   if start < 0 || start >= len(videoData) {
      return nil, fmt.Errorf("start byte %d out of range", start)
   }
  
   if end < start || end >= len(videoData) {
      end = len(videoData) - 1 // Adjust end to the last byte if it's out of bounds
   }
  
   // Return the requested slice of video data
   return videoData[start : end+1], nil
}

The next step will be to add an HTTP handler that will expose all this neat code to the network.

HTTP Handler

For this post, I’ll be using the HandlerFunc type from the standard library’s http package to perform the streaming. It’s a vanilla setup per se. The HandlerFunc will be generated through a factory function. This factory function will have three parameters:

• streamer: This will be of type VideoStreamer and it is used by the handler to fetch the requested parts of a file.
• videoKey: the unique ID of the video. This will be passed as the key parameter of the Seek method.
• totalSize: the file size of the video.

I’ll break the HandlerFunc functionality into three parts:

• Read and prepare the data received from the client request.
• Load the bytes requested by the client.
• Send the bytes requested, with the appropriate headers, back to the client.

During the first part, my handler will read the value of the Range header. Once read, the handler will extract the start and end index of the bytes requested. This extraction will be done by splitting the Range header value and converting the start and end index strings into type int. Here is the code for the first part:

func VideoStreamHandler(streamer VideoStreamer, videoKey string, totalSize int) http.HandlerFunc {
   return func(w http.ResponseWriter, r *http.Request) {
  
      rangeHeader := r.Header.Get("Range")
      var start, end int
    
      if rangeHeader == "" {
         // Default to first 1MB
         start = 0
         end = 1024*1024 - 1
      } else {
    
         // Parse the Range header: "bytes=start-end"
         rangeParts := strings.TrimPrefix(rangeHeader, "bytes=")
         rangeValues := strings.Split(rangeParts, "-")
         var err error
      
         // Get start byte
         start, err = strconv.Atoi(rangeValues[0])
         if err != nil {
            http.Error(w, "Invalid start byte", http.StatusBadRequest)
            return
         }
      
         // Get end byte or set to default i
         if len(rangeValues) > 1 && rangeValues[1] != "" {
            end, err = strconv.Atoi(rangeValues[1])
            if err != nil {
               http.Error(w, "Invalid end byte", http.StatusBadRequest)
               return
            }
         } else {
            end = start + 1024*1024 - 1 // Default 1MB chunk
         }
      }
    
      // Ensure end is within the total video size
      if end >= totalSize {
         end = totalSize - 1
      }
  
      ...
   }
}

For the second part, I’ll pass the videoKey,start and end variables to the VideoStreamer‘s Seek method to get the bytes requested. Here is the code performing this:

  func VideoStreamHandler(streamer VideoStreamer, videoKey string, totalSize int) http.HandlerFunc {
   return func(w http.ResponseWriter, r *http.Request) {
  
      ...
      // Fetch the video data
      videoData, err := streamer.Seek(videoKey, start, end)
      if err != nil {
         http.Error(w, fmt.Sprintf("Error retrieving video: %v", err), http.StatusInternalServerError)
         return
      }
    
      ...
   }
}

Once the bytes requested is loaded onto memory, I can begin the third part and return a response to the client. The first thing I’ll do is return a set of headers that are crucial for streaming to work, without any additional HTML… here is the list of headers:

• Content-Range: Indicates the byte range and total size of the file being served in the response.
• Accept-Ranges: Indicates whether the server supports range requests.
• Content-Length: Specifies the size of the response body in bytes.
• Content-Type: Specifies the media type of the resource being served. This will ensure that the client (e.g., a browser or media player) interprets the content correctly.

Next, I’ll set the response status code to Partial Content (206) and write the bytes returned by the VideoStreamer interface as the response body.

func VideoStreamHandler(streamer VideoStreamer, videoKey string, totalSize int) http.HandlerFunc {
   return func(w http.ResponseWriter, r *http.Request) {
      ...
      // Set headers and serve the video chunk
      w.Header().Set("Content-Range", fmt.Sprintf("bytes %d-%d/%d", start, end, totalSize))
      w.Header().Set("Accept-Ranges", "bytes")
      w.Header().Set("Content-Length", fmt.Sprintf("%d", len(videoData)))
      w.Header().Set("Content-Type", "video/mp4")
      w.WriteHeader(http.StatusPartialContent)

      _, err = w.Write(videoData)
      if err != nil {
         http.Error(w, "Error streaming video", http.StatusInternalServerError)
      }
   }
}

You can find the complete source code of this handler in the Sources section below.

Putting It Together

To test the previously defined code, I’ll add a main function that does the following:

• Download a MP4 from a stock website. Then, construct an instance of type MockVideoStreamer to store the downloaded MP4 in.
• Bind the HandlerFunc, returned by VideoStreamHandler, to route /video.
• Start a web server listening on port 8080.

Here is the code performing the tasks outlined above:

const url = "https://download.samplelib.com/mp4/sample-5s.mp4"

func main() {
    
   // DownloadBytes is a util. function.
   // See full source code in Sources section
   // below
   data, err := DownloadBytes(url)
   if err != nil {
      fmt.Println("Error downloading:", err)
      return
   }
  
   log.Println("length of data:", len(data))
   streamer := &MockVideoStreamer{
    Store: map[string][]byte{"video-key": data},
   }
  
   http.HandleFunc("/video", VideoStreamHandler(streamer, "video-key", len(data)))
   http.ListenAndServe(":8080", nil)
}

And… here it is in action:

Network throttled to 3G

Conclusion

This post demonstrates how client range requests can be leveraged to deliver a streaming experience. It also represents what Go can do out of the box. The code in this post also has its set of limitations:

• It does not support multiple ranges.
• It loads file data onto memory. The correct approach would be to stream the data back with io.Reader.
• No support for full content requests, only portions.

Streaming in environments with high traffic may require a different setup. I do not recommend using any of this code in such environments, especially due to the limitations outlined above.

Thank you for reading.

Sources

RFC 7233 — https://datatracker.ietf.org/doc/html/rfc7233

Complete code — https://gist.github.com/cheikhsimsol/12183f73b250bc09952ae52b5f36f53e