Automation Testing with Streamlit

A robotframework & selenium way to test streamlit application

Streamlit is a very good tool and it is used to create demo application, but sometimes this demo come into production environment, so it is important to find methods to test it. Into recent micro-front-end pattern it is more probable that a streamlit app become a part of the production server, and in the future it can be changed with a part of mircro-front-end.

In this story I explain how I test a toy streamlit app with robotframework and I automate the pipelnine on gitlab, the code is available here.
(this story is from zero/step-by-step, but to understand more you must read about libraries used and you must read also about automation test)

I chouse robotframework because I want to learn it, but it is possible to do the same wich test libraries you want, the simplest is pytest, but you can also use mamba and more.

Toy App

We need to install streamlit only

pip install streamlit

The toy app is a simple application that have a radio button, in witch if you selecte Commey option it show ‘You selected comedy.’ elsewere it show “You didn’t select comedy.”:

import streamlit as st

st.title("Test")
genre = st.radio(
    "What's your favorite movie genre",
    ('Comedy', 'Drama', 'Documentary'))

if genre == 'Comedy':
    st.write('You selected comedy.')
else:
    st.write("You didn't select comedy.")

now we can save it into src/app.py and run it

python3.8 -m streamlit run src/app.py --server.port 5000

Toy streamlit app

Requirements

We need some other libraries, robotframework to test, selenium and the robot framework binding to use the browser by code and coverage to obtain the test coverage (how many lines of code our test cover):

pip install robotframework robotframework-SeleniumLibrary selenium coverage

Now we need to add a driver for controlling our browser, I select Firefox so we need geckodriver we can install it by copiing into bin (on Linux)

sudo cp driver/geckodriver /bin/
sudo chmod +x /bin/geckodriver

Or you can specify environment variable only for the terminal that you will use to run the test

export PATH=$PATH:/home/opensuse/Projects/streamlit_test/driver/.

Test

Now we are able to write tests under test/app.robot

*** Settings ***
Library  SeleniumLibrary
Library  Process

Suite Setup         Start the webserver

Suite Teardown      Stop the webserver

*** Keywords ***
Start the webserver
    Log To Console  start
    ${process}=     Start Process       python3     -m      coverage    run     --source    src    -m      streamlit    run     src/app.py     --server.port       5000    --server.headless   true

    Set suite variable    ${process}
    Log To Console     ${process}
    sleep  2s

Stop the webserver
    Log To Console  end
    Terminate Process    ${process}


*** Variables ***
${URL}             http://localhost:5000/
${BROWSER}         headlessfirefox

*** Test Cases ***


first test
    Log To Console  test1
    Open Browser  ${URL}  browser=${BROWSER}
    Wait Until Page Contains    Test
    Page Should Contain     You selected comedy.
    Click Element       //*[contains(text(),'Drama')]
    sleep   1
    Page Should Contain     You didn't select comedy.
    Close Browser

This script run the streamlit app by command (through coverage, to compute code coverage) line and after go on headlessfirefox (you can change it to firefox to view what happen on your screen) at the streamlit url and wait the app show The title`Test` after it say that `You selected comedy.` is present, after click a different radio button (using an XPath expression to find radio button into HTML) wait and assert that the page contain `You didn’t select comedy.` after close the browser and stop the streamlit app.

Now we can run with:

python3.8 -m robot test

to obtain the code coverage:

python3.8 -m coverage report

If the assert not pass it return error otherwise the test pass.

Continous Integration Pipeline

I put the requirements into requirements.txt

streamlit
robotframework
robotframework-SeleniumLibrary
selenium
coverage

To run this test each time you push on your gitlab repository you can create a .gitlab-ci.yml file like the following:

robotframework:
    image:
        name: selenium/standalone-firefox:99.0
    script:
        - sudo apt-get update && sudo apt-get install -y python3 python3-pip
        - pip install --upgrade pip && pip install -r requirements.txt
        - python3 -m robot test
        - python3 -m coverage report
    coverage: /^TOTAL.*\s+(\d+\%)$/

It specify a container that have selenium and firefox, install python and requirements, run test and show coverage. The regex at the end extract the percent coverage and say to gitlab inorder to have badges.

Pipeline passed and coverage gitlab badges

Conclusions

This test simulate the user interactions with our streamlit app, are the UI test on top of test piramyd so that tests are the much expensive ones, but it can be very usefull to ensure the correctenss of our program also throw changes and in more than one browser: this example is only on firefox but you can run the same tests on chrome and on each browser you want.

So in this way streamlit can be used also as support for micro-front-end application, and can go in production safelly.

Remember that this is possible thanks open source so contribute!