Deploy a Streamlit App to AWS

a incredible Streamlit app, and now it’s time to let the world see and use it. 

What choices do you may have? 

The best method is to make use of the Streamlit Neighborhood Cloud service. That technique lets anybody on-line entry your Streamlit app, supplied they’ve the required URL. It’s a comparatively easy course of, however it’s a publicly out there endpoint and, attributable to potential safety points and scalability choices, it isn’t an possibility for many organisations.

Since Streamlit was acquired by Snowflake, deploying to that platform is now a viable possibility as effectively. 

The third possibility is to deploy to one of many many cloud companies, resembling Heroku, Google Cloud, or Azure.

As an AWS consumer, I needed to see how straightforward it might be to deploy a streamlit app to AWS, and that is what this text is about. When you confer with the official Streamlit documentation on-line (hyperlink on the finish of the article), you’ll discover that there isn’t any info or steering on how to do that. So that is the “lacking handbook”.

The deployment course of is comparatively easy. The difficult half is making certain that the AWS networking configuration is ready up appropriately. By that, I imply your VPC, safety teams, subnets, route tables, subnet associations, Nat Gateways, Elastic IPS, and so on…

As a result of each organisation’s networking setup is completely different, I’ll assume that you simply or somebody in your organisation can resolve this side. Nonetheless, I embody some troubleshooting ideas on the finish of the article for the commonest causes for deployment points. When you comply with my steps to the letter, you ought to have a working, deployed app by the tip of it.

In my pattern deployment, I’ll be utilizing a VPC with a Public subnet and an Web gateway. Against this, in real-life situations, you’ll most likely need to use a mixture of all or a few of elastic load balancers, non-public subnets, NAT gateways and Cognito for consumer authentication and enhanced safety. Afterward, I’ll talk about some choices for securing your app.

The app we’ll deploy is the dashboard I wrote utilizing Streamlit. TDS revealed that article some time again, and yow will discover a hyperlink to it on the finish of this text. In that case, I retrieved my dashboard information from a PostgreSQL database operating regionally. Nonetheless, to keep away from the prices and problem of organising an RDS Postgres database on AWS, I’ll convert my dashboard code to retrieve its information from a CSV file on S3 — Amazon’s mass storage service. 

As soon as that’s finished, it’s solely a matter of copying over a CSV to AWS S3 storage, and the dashboard ought to work simply because it did when operating regionally utilizing Postgres.

I assume you may have an AWS account with entry to the AWS console. Moreover, if you’re choosing the S3 route as your information supply, you’ll must arrange AWS credentials. Upon getting them, both create an .aws/credentials file in your HOME listing (as I’ve finished), or you may go your credential key info instantly within the code.

Assuming all these stipulations are met, we will take a look at the deployment utilizing AWS’s Elastic Beanstalk service.

What’s AWS Elastic Beanstalk (EB)?

AWS Elastic Beanstalk (EB) is a totally managed service that simplifies the deployment, scaling, and administration of purposes within the AWS Cloud. It means that you can add your utility code in fashionable languages like Python, Java, .NET, Node.js, and extra. It routinely handles the provisioning of the underlying infrastructure, resembling servers, load balancers, and networking. With Elastic Beanstalk, you may deal with writing and sustaining your utility moderately than configuring servers or managing capability as a result of the service seamlessly scales assets as your utility’s site visitors fluctuates.

Along with provisioning your EC2 servers, and so on., EB will set up any required exterior libraries in your behalf, relying on the deployment sort. It may also be configured to run OS instructions on server startup.

The code

Earlier than deploying, let’s evaluate the adjustments I made to my authentic code to accommodate the change in information supply from Postgres to S3. It boils all the way down to changing calls to learn a Postgres desk with calls to learn an S3 object to feed information into the dashboard. I additionally put the primary graphical element creation and show inside a fundamental() module, which I name on the finish of the code. Here’s a full itemizing.

import streamlit as st
import pandas as pd
import matplotlib.pyplot as plt
import datetime
import boto3
from io import StringIO

#########################################
# 1. Load Information from S3
#########################################

@st.cache_data
def load_data_from_s3(bucket_name, object_key):
    """
    Reads a CSV file from S3 right into a Pandas DataFrame.
    Be sure that your AWS credentials are correctly configured.
    """
    s3 = boto3.consumer("s3")
    obj = s3.get_object(Bucket=bucket_name, Key=object_key)
    df = pd.read_csv(obj['Body'])
    
    # Convert order_date to datetime if wanted
    df['order_date'] = pd.to_datetime(df['order_date'], format='%d/%m/%Y')
    
    return df

#########################################
# 2. Helper Capabilities (Pandas-based)
#########################################

def get_date_range(df):
    """Return min and max dates within the dataset."""
    min_date = df['order_date'].min()
    max_date = df['order_date'].max()
    return min_date, max_date

def get_unique_categories(df):
    """
    Return a sorted listing of distinctive classes (capitalized).
    """
    classes = df['categories'].dropna().distinctive()
    classes = sorted([cat.capitalize() for cat in categories])
    return classes

def filter_dataframe(df, start_date, end_date, class):
    """
    Filter the dataframe by date vary and optionally by a single class.
    """
    # Guarantee begin/end_date are transformed to datetime simply in case
    start_date = pd.to_datetime(start_date)
    end_date = pd.to_datetime(end_date)
    
    masks = (df['order_date'] >= start_date) & (df['order_date'] <= end_date)
    filtered = df.loc[mask].copy()
    
    # If not "All Classes," filter additional by class
    if class != "All Classes":
        # Classes in CSV is perhaps lowercase, uppercase, and so on.
        # Modify as wanted to match your information
        filtered = filtered[filtered['categories'].str.decrease() == class.decrease()]
    
    return filtered

def get_dashboard_stats(df, start_date, end_date, class):
    """
    Calculate whole income, whole orders, common order worth, and high class.
    """
    filtered_df = filter_dataframe(df, start_date, end_date, class)
    if filtered_df.empty:
        return 0, 0, 0, "N/A"
    
    filtered_df['revenue'] = filtered_df['price'] * filtered_df['quantity']
    total_revenue = filtered_df['revenue'].sum()
    total_orders = filtered_df['order_id'].nunique()
    avg_order_value = total_revenue / total_orders if total_orders > 0 else 0
    
    # Decide high class by whole income
    cat_revenue = filtered_df.groupby('classes')['revenue'].sum().sort_values(ascending=False)
    top_cat = cat_revenue.index[0].capitalize() if not cat_revenue.empty else "N/A"
    
    return total_revenue, total_orders, avg_order_value, top_cat

def get_plot_data(df, start_date, end_date, class):
    """
    For 'Income Over Time', group by date and sum income.
    """
    filtered_df = filter_dataframe(df, start_date, end_date, class)
    if filtered_df.empty:
        return pd.DataFrame(columns=['date', 'revenue'])
    
    filtered_df['revenue'] = filtered_df['price'] * filtered_df['quantity']
    plot_df = (
        filtered_df.groupby(filtered_df['order_date'].dt.date)['revenue']
        .sum()
        .reset_index()
        .rename(columns={'order_date': 'date'})
        .sort_values('date')
    )
    return plot_df

def get_revenue_by_category(df, start_date, end_date, class):
    """
    For 'Income by Class', group by class and sum income.
    """
    filtered_df = filter_dataframe(df, start_date, end_date, class)
    if filtered_df.empty:
        return pd.DataFrame(columns=['categories', 'revenue'])
    
    filtered_df['revenue'] = filtered_df['price'] * filtered_df['quantity']
    rev_cat_df = (
        filtered_df.groupby('classes')['revenue']
        .sum()
        .reset_index()
        .sort_values('income', ascending=False)
    )
    rev_cat_df['categories'] = rev_cat_df['categories'].str.capitalize()
    return rev_cat_df

def get_top_products(df, start_date, end_date, class, top_n=10):
    """
    For 'Prime Merchandise', return high N merchandise by income.
    """
    filtered_df = filter_dataframe(df, start_date, end_date, class)
    if filtered_df.empty:
        return pd.DataFrame(columns=['product_names', 'revenue'])
    
    filtered_df['revenue'] = filtered_df['price'] * filtered_df['quantity']
    top_products_df = (
        filtered_df.groupby('product_names')['revenue']
        .sum()
        .reset_index()
        .sort_values('income', ascending=False)
        .head(top_n)
    )
    return top_products_df

def get_raw_data(df, start_date, end_date, class):
    """
    Return the uncooked (filtered) information with a income column.
    """
    filtered_df = filter_dataframe(df, start_date, end_date, class)
    if filtered_df.empty:
        return pd.DataFrame()
    
    filtered_df['revenue'] = filtered_df['price'] * filtered_df['quantity']
    filtered_df = filtered_df.sort_values(by=['order_date', 'order_id'])
    return filtered_df

def plot_data(information, x_col, y_col, title, xlabel, ylabel, orientation='v'):
    fig, ax = plt.subplots(figsize=(10, 6))
    if not information.empty:
        if orientation == 'v':
            ax.bar(information[x_col], information[y_col])
            plt.xticks(rotation=45)
        else:
            ax.barh(information[x_col], information[y_col])
        ax.set_title(title)
        ax.set_xlabel(xlabel)
        ax.set_ylabel(ylabel)
    else:
        ax.textual content(0.5, 0.5, "No information out there", ha='heart', va='heart')
    return fig

#########################################
# 3. Streamlit Utility
#########################################

def fundamental():
    # Title
    st.title("Gross sales Efficiency Dashboard")

    # Load your information from S3
    # Exchange these together with your precise bucket identify and object key
    bucket_name = "your_s3_bucket_name"
    object_key = "your_object_name"
    
    df = load_data_from_s3(bucket_name, object_key)
    
    # Get min and max date for default vary
    min_date, max_date = get_date_range(df)

    # Create UI for date and class filters
    with st.container():
        col1, col2, col3 = st.columns([1, 1, 2])
        start_date = col1.date_input("Begin Date", min_date)
        end_date = col2.date_input("Finish Date", max_date)
        classes = get_unique_categories(df)
        class = col3.selectbox("Class", ["All Categories"] + classes)

    # Customized CSS for metrics
    st.markdown("""
        
    """, unsafe_allow_html=True)

    # Fetch stats
    total_revenue, total_orders, avg_order_value, top_category = get_dashboard_stats(df, start_date, end_date, class)

    # Show key metrics
    metrics_html = f"""
    

        

            
Complete Income
            
${total_revenue:,.2f}
        
        

            
Complete Orders
            
{total_orders:,}
        
        

            
Common Order Worth
            
${avg_order_value:,.2f}
        
        

            
Prime Class
            
{top_category}
        
    
    """
    st.markdown(metrics_html, unsafe_allow_html=True)

    # Visualization Tabs
    st.header("Visualizations")
    tabs = st.tabs(["Revenue Over Time", "Revenue by Category", "Top Products"])

    # Income Over Time Tab
    with tabs[0]:
        st.subheader("Income Over Time")
        revenue_data = get_plot_data(df, start_date, end_date, class)
        st.pyplot(plot_data(revenue_data, 'date', 'income', "Income Over Time", "Date", "Income"))

    # Income by Class Tab
    with tabs[1]:
        st.subheader("Income by Class")
        category_data = get_revenue_by_category(df, start_date, end_date, class)
        st.pyplot(plot_data(category_data, 'classes', 'income', "Income by Class", "Class", "Income"))

    # Prime Merchandise Tab
    with tabs[2]:
        st.subheader("Prime Merchandise")
        top_products_data = get_top_products(df, start_date, end_date, class)
        st.pyplot(plot_data(top_products_data, 'product_names', 'income', "Prime Merchandise", "Income", "Product Identify", orientation='h'))

    # Uncooked Information
    st.header("Uncooked Information")
    raw_data = get_raw_data(df, start_date, end_date, class)
    raw_data = raw_data.reset_index(drop=True)
    st.dataframe(raw_data, hide_index=True)

if __name__ == '__main__':
    fundamental()

Though it’s a reasonably chunky piece of code, I gained’t clarify precisely what it does, as I’ve already lined that in some element in my beforehand referenced TDS article. I’ve included a hyperlink to the article on the finish of this one for individuals who want to study extra.

So, assuming you may have a working Streamlit app that runs regionally with out points, listed below are the steps it is advisable take to deploy it to AWS.

Making ready our code for deployment

1/ Create a brand new folder in your native system to carry your code.

2/ In that folder, you’ll want three information and a sub-folder containing two extra information

• File 1 is app.py —  that is your fundamental Streamlit code file
• File 2 is necessities.txt — this lists all exterior libraries your code must operate. Relying on what your code does, it should have no less than one report referencing the Streamlit library. For my code, the file contained this,

streamlit
boto3
matplotlib
pandas

• File 3 known as Procfile — this tells EB methods to run your code. It’s contents ought to seem like this

net: streamlit run app.py --server.port 8000 --server.enableCORS false

• .ebextensions — it is a subfolder which holds further information (see beneath)

3/ The .ebextensions subfolder has these two information.

It ought to have this content material:

option_settings:
  aws:elasticbeanstalk:surroundings:proxy:
    ProxyServer: nginx

option_settings:
  aws:elasticbeanstalk:container:python:
    WSGIPath: app:fundamental

Be aware, though I didn’t want it for what I used to be doing, for completenes, you may optionally add a number of packages.config information beneath the .ebextensions subfolder that may comprise working system instructions which are run when the EC2 server begins up. For instance, 

#
# 01_packages.config
#
packages:
    yum:
        amazon-linux-extras: []

instructions:
    01_postgres_activate:
        command: sudo amazon-linux-extras allow postgresql10
    02_postgres_install:
        command: sudo yum set up -y pip3
    03_postgres_install:
        command: sudo pip3 set up -y psycopg2

Upon getting all the mandatory information, the subsequent step is to zip them into an archive, preserving the folder and subfolder construction. You should use any software you want, however I exploit 7-Zip.

Deploying our code

Deployment is a multi-stage course of. First, log in to the AWS console, seek for “Elastic Beanstalk” within the companies search bar, and click on on the hyperlink. From there, you may click on the massive orange “Create Utility” button. You’ll see the primary of round six screens, for which you need to fill within the particulars. Within the following sections, I’ll describe the fields you need to enter. Go away the whole lot else as it’s.

1/ Creating the applying

• That is straightforward: fill within the identify of your utility and, optionally, its description.

2/ Configure Surroundings

• The surroundings tier ought to be set to Internet Server.
• Fill within the utility identify.
• For Platform sort, select Managed; for Platform, select Python, then resolve which model of Python you need to use. I used Python model 3.11.
• Within the Utility Code part, click on the Add your code possibility and comply with the directions. Sort in a model label, then click on ‘Native File’ or ‘S3 Add’, relying on the place your supply information are situated. You need to add the only zip file we created earlier.
• Select your occasion sort within the Presets part. I went for the Single occasion (free tier eligible). Then hit the Subsequent button.

3/ Configure Service Entry

• For the Service function, you should utilize an present one in case you have it, or AWS will create one for you.
• For the occasion profile function, you’ll most likely must create this. It simply must have the AWSElasticBeanstalkWebTier and AmazonS3ReadOnlyAccess insurance policies connected. Hit the Subsequent button.
• I might additionally advise organising an EC2 key pair at this stage, as you’ll want it to log in to the EC2 server that EB creates in your behalf. This may be invaluable for investigating potential server points.

4/ Arrange networking, database and tags

• Select your VPC. I had just one default VPC arrange. You even have the choice to create one right here when you don’t have already got one. Be sure that your VPC has no less than one public subnet.
• In Occasion Settings, I checked the Public IP Handle possibility, and I selected to make use of my public subnets. Click on the Subsequent button.

5/ Configure the occasion and scaling

• Below the EC2 Safety Teams part, I selected my default safety group. Below Occasion Sort, I opted for the t3.micro. Hit the Next button.

6/ Monitoring

• Choose primary system well being monitoring
• Uncheck the Managed Updates checkbox
• Click on Subsequent

7/ Assessment

• Click on Create if all is OK

After this, it is best to see a display like this,

Regulate the Occasions tab, as this can notify you if any points come up. When you encounter issues, you should utilize the Logs tab to retrieve both a full set of logs or the final 100 strains of the deployment log, which can assist you debug any points. 

After a couple of minutes, if all has gone effectively, the Well being label will swap from gray to inexperienced and your display will look one thing like this:

Now, it is best to have the ability to click on on the Area URL (circled in crimson above), and your dashboard ought to seem.

Troubleshooting

The very first thing to verify when you encounter issues when operating your dashboard is that your supply information is within the right location and is referenced appropriately in your Streamlit app supply code file. When you rule that out as a problem, then you’ll greater than probably have hit a networking setup drawback, and also you’ll most likely see a display like this.

If that’s the case, right here are some things you may take a look at. Chances are you’ll must log in to your EC2 occasion and evaluate the logs. In my case, I encountered a problem with my pip set up command, which ran out of house to put in all the mandatory packages. To resolve that, I had so as to add additional Elastic Block storage to my occasion. 

The extra probably trigger will probably be a networking concern. In that case, attempt some or all the recommendations beneath.

VPC Configuration

• Guarantee your Elastic Beanstalk surroundings is deployed in a VPC with no less than one public subnet.
• Confirm that the VPC has an Web Gateway connected.

Subnet Configuration

• Verify that the subnet utilized by your Elastic Beanstalk surroundings is public.
• Verify that the “Auto-assign public IPv4 handle” setting is enabled for this subnet.

Route Desk

• Confirm that the route desk related together with your public subnet has a path to the Web Gateway (0.0.0.0/0 -> igw-xxxxxxxx).

Safety Group

• Assessment the inbound guidelines of the safety group connected to your Elastic Beanstalk cases.
• Guarantee it permits incoming site visitors on port 80 (HTTP) and/or 443 (HTTPS) from the suitable sources.
• Verify that outbound guidelines enable needed outgoing site visitors.

Community Entry Management Lists (NACLS)

• Assessment the Community ACLS related together with your subnet.
• Guarantee they permit each inbound and outbound site visitors on the mandatory ports.

Elastic Beanstalk Surroundings Configuration

• Confirm that your surroundings is utilizing the right VPC and public subnet within the Elastic Beanstalk console.

EC2 Occasion Configuration

• Confirm that the EC2 cases launched by Elastic Beanstalk have public IP addresses assigned.

Load Balancer Configuration (if relevant)

• When you use a load balancer, guarantee it’s configured appropriately within the public subnet.
• Verify that the load balancer safety group permits incoming site visitors and might talk with the EC2 cases.

Securing your app

Because it stands, your deployed app is seen to anybody on the web who is aware of your deployed EB area identify. That is most likely not what you need. So, what are your choices for securing your app on AWS infrastructure?

1/ Lock the safety group to trusted CIDRs

Within the console, discover the safety group related together with your EB deployment and click on on it. It ought to seem like this,

Be sure you’re on the Inbound Guidelines TAB, select Edit Inbound Guidelines, and alter the supply IP ranges to your company IP ranges or one other set of IP addresses.

2/ Use non-public subnets, inner load balancers and NAT Gateways

It is a more difficult choice to implement and sure requires the experience of your AWS community administrator or deployment specialist.

3/ Utilizing AWS Cognito and an utility load balancer

Once more, it is a extra complicated setup that you simply’ll most likely want help with when you’re not an AWS community guru, however it’s maybe essentially the most sturdy of all of them. The stream is that this:-

A consumer navigates to your public Streamlit URL.

The ALB intercepts the request. It sees that the consumer is both not already logged in or not authenticated.

The ALB routinely redirects the consumer to Cognito to sign up or create an account. Upon profitable login, Cognito redirects the consumer again to your utility URL. The ALB now recognises a sound session and permits the request to proceed to your Streamlit app.

Your Streamlit app solely ever receives site visitors from authenticated customers.

Abstract

On this article, I mentioned deploying a Streamlit dashboard utility I had beforehand written to AWS. The unique app utilised PostgreSQL as its information supply, and I demonstrated methods to swap to utilizing AWS S3 in preparation for deploying the app to AWS. 

I mentioned deploying the app to AWS utilizing their Elastic Beanstalk service. I described and defined all the additional information required earlier than deployment, together with the necessity for them to be contained in a zipper archive.

I then briefly defined the Elastic Beanstalk service and described the detailed steps required to make use of it to deploy our Streamlit app to AWS infrastructure. I described the a number of enter screens that wanted to be navigated and confirmed what inputs to make use of at varied phases.

I highlighted some troubleshooting strategies if the app deployment doesn’t go as anticipated.

Lastly, I supplied some recommendations on methods to shield your app from unauthorised entry.

For extra info on Streamlit, take a look at their on-line documention utilizing the hyperlink beneath.

https://docs.streamlit.io

To seek out out extra about creating with Streamlit I present methods to develop a contemporary information dashboard with it within the article linked beneath.