Siv Scripts

Solving Problems Using Code

Thu 17 August 2017

Interactive, Web-Based Dashboards in Python

Posted by Aly Sivji in Tutorials   

Summary

  • Explore Plotly's new Dash library
  • Discuss how to structure Dash apps using MVC
  • Build interactive dashboard to display historical soccer results

I spent a good portion of 2014-15 learning JavaScript to create interactive, web-based dashboards for a work project. I wrapped D3.js with Angular directives to create modular components that were used to visualize data.

Data Analysis is not one of JavaScript's strengths; most of my code was trying to cobble together DataFrame-esque operations with JSON data. I missed R. I missed Python. I even missed MATLAB.

When I found Dash a couple of months ago, I was blown away.

With Dash, we can create interactive, web-based dashboards with pure Python. All the front-end work, all that dreaded JavaScript, that's not our problem anymore.

How easy is Dash to use? In around an hour and with <100 lines of code, I created a dashboard to display live streaming data for my Data Science Workflows using Docker Containers talk.

Dash is a powerful library that simplifies the development of data-driven applications. Dash enables Data Scientists to become Full Stack Developers.

In this post we will explore Dash, discuss how the Model-View-Controller pattern can be used to structure Dash applications, and build a dashboard to display historical soccer results.


Dash Overview

Dash is a Open Source Python library for creating reactive, Web-based applications

Dash apps consist of a Flask server that communicates with front-end React components using JSON packets over HTTP requests.

What does this mean? We can run a Flask app to create a web page with a dashboard. Interaction in the browser can call code to re-render certain parts of our page.

We use the provided Python interface to design our application layout and to enable interaction between components. User interaction triggers Python functions; these functions can perform any action before returning a result back to the specified component.

Dash applications are written in Python. No HTML or JavaScript is necessary.

We are also able to plug into React's extensive ecosystem through an included toolset that packages React components into Dash-useable components.


Dash Application Design: MVC Pattern

As I worked my way through the documentation, I kept noticing that every Dash application could be divided into the following components:

  • Data Manipulation - Perform operations to read / transform data for display
  • Dashboard Layout - Visually render data into output representation
  • Interaction Between Components - Convert user input to commands for data manipulation + render

Good god, that's the Undertaker's Model-View-Controller (MVC) Pattern's music! (Note: I covered MVC in a previous post)

When designing a Dash application, we should stucture our code into three sections:

  1. Data Manipulation (Model)
  2. Dashboard Layout (View)
  3. Interaction Between Components (Controller)

I created the following template to help us get started:


Historical Match-up Dashboard

In this section, we will create a full-featured Dash application that can be used to view historical soccer data.

We will use the following process to create / modify Dash applications:

  1. Create/Update Layout - Figure out where components will be placed
  2. Map Interactions with Other Components - Specify interaction in callback decorators
  3. Wire in Data Model - Data manipulation to link interaction and render

Setting Up Environment and Installing Dependencies

There are installation instructions in the Dash Documentation. Alternatively, we can create a virtualenv and pip install the requirements file.

mkdir historical-results-dashboard && cd historical-results-dashboard
mkvirtualenv dash-app
wget https://raw.githubusercontent.com/alysivji/historical-results-dashboard/master/requirements.txt
pip install -r requirements.txt

Data is stored in an SQLite database:

wget https://github.com/alysivji/historical-results-dashboard/blob/master/soccer-stats.db?raw=true soccer-stats.db

Download the Dash application template file from above:

wget https://gist.githubusercontent.com/alysivji/e85a04f3a9d84f6ce98c56f05858ecfb/raw/d7bfeb84e2c825cfb5d4feee15982c763651e72e/dash_app_template.py app.py

Dashboard Layout (View)

Our app will look as follows:
Dash App Layout

Users will be able to select Division, Season, and Team via Dropdown components. Selection will trigger actions to update tables (Results + Win/Loss/Draw/Points Summary) and a graph (Points Accumulated vs Time)

We begin by translating the layout from above into Dash components (both core + HTML components will be required):

#########################
# Dashboard Layout / View
#########################

def generate_table(dataframe, max_rows=10):
    '''Given dataframe, return template generated using Dash components
    '''
    return html.Table(
        # Header
        [html.Tr([html.Th(col) for col in dataframe.columns])] +

        # Body
        [html.Tr([
            html.Td(dataframe.iloc[i][col]) for col in dataframe.columns
        ]) for i in range(min(len(dataframe), max_rows))]
    )


def onLoad_division_options():
    '''Actions to perform upon initial page load'''

    division_options = (
        [{'label': division, 'value': division}
         for division in get_divisions()]
    )
    return division_options


# Set up Dashboard and create layout
app = dash.Dash()
app.css.append_css({
    "external_url": "https://codepen.io/chriddyp/pen/bWLwgP.css"
})

app.layout = html.Div([

    # Page Header
    html.Div([
        html.H1('Soccer Results Viewer')
    ]),

    # Dropdown Grid
    html.Div([
        html.Div([
            # Select Division Dropdown
            html.Div([
                html.Div('Select Division', className='three columns'),
                html.Div(dcc.Dropdown(id='division-selector',
                                      options=onLoad_division_options()),
                         className='nine columns')
            ]),

            # Select Season Dropdown
            html.Div([
                html.Div('Select Season', className='three columns'),
                html.Div(dcc.Dropdown(id='season-selector'),
                         className='nine columns')
            ]),

            # Select Team Dropdown
            html.Div([
                html.Div('Select Team', className='three columns'),
                html.Div(dcc.Dropdown(id='team-selector'),
                         className='nine columns')
            ]),
        ], className='six columns'),

        # Empty
        html.Div(className='six columns'),
    ], className='twleve columns'),

    # Match Results Grid
    html.Div([

        # Match Results Table
        html.Div(
            html.Table(id='match-results'),
            className='six columns'
        ),

        # Season Summary Table and Graph
        html.Div([
            # summary table
            dcc.Graph(id='season-summary'),

            # graph
            dcc.Graph(id='season-graph')
            # style={},

        ], className='six columns')
    ]),
])
Notes:
  • We used HTML <DIV> elements and the Dash Style Guide to design the layout
  • Tables can be rendered two different ways: Native HTML or Plotly Table
  • We just wire-framed components in this section, data will be populated via Model and Controller

Interaction Between Components (Controller)

Once we create our layout, we will need to map out the interaction between the various components. We do this using the provided app.callback() decorator.

The parameters we pass into the decorator are:

  • Output component + property we want to update
  • list of all the Input components + properties that can be used to trigger the function

Our code looks as follows:

#############################################
# Interaction Between Components / Controller
#############################################

# Load Seasons in Dropdown
@app.callback(
    Output(component_id='season-selector', component_property='options'),
    [
        Input(component_id='division-selector', component_property='value')
    ]
)
def populate_season_selector(division):
    seasons = get_seasons(division)
    return [
        {'label': season, 'value': season}
        for season in seasons
    ]


# Load Teams into dropdown
@app.callback(
    Output(component_id='team-selector', component_property='options'),
    [
        Input(component_id='division-selector', component_property='value'),
        Input(component_id='season-selector', component_property='value')
    ]
)
def populate_team_selector(division, season):
    teams = get_teams(division, season)
    return [
        {'label': team, 'value': team}
        for team in teams
    ]


# Load Match results
@app.callback(
    Output(component_id='match-results', component_property='children'),
    [
        Input(component_id='division-selector', component_property='value'),
        Input(component_id='season-selector', component_property='value'),
        Input(component_id='team-selector', component_property='value')
    ]
)
def load_match_results(division, season, team):
    results = get_match_results(division, season, team)
    return generate_table(results, max_rows=50)


# Update Season Summary Table
@app.callback(
    Output(component_id='season-summary', component_property='figure'),
    [
        Input(component_id='division-selector', component_property='value'),
        Input(component_id='season-selector', component_property='value'),
        Input(component_id='team-selector', component_property='value')
    ]
)
def load_season_summary(division, season, team):
    results = get_match_results(division, season, team)

    table = []
    if len(results) > 0:
        summary = calculate_season_summary(results)
        table = ff.create_table(summary)

    return table


# Update Season Point Graph
@app.callback(
    Output(component_id='season-graph', component_property='figure'),
    [
        Input(component_id='division-selector', component_property='value'),
        Input(component_id='season-selector', component_property='value'),
        Input(component_id='team-selector', component_property='value')
    ]
)
def load_season_points_graph(division, season, team):
    results = get_match_results(division, season, team)

    figure = []
    if len(results) > 0:
        figure = draw_season_points_graph(results)

    return figure
Notes
  • Each app.callback() decorator can be bound to a single Output (component, property) pair
    • We will need to create additional functions to change multiple Output components
  • We could add Data Manipulation code in this section, but separating the app into components makes it easier to work with

Data Manipulation (Model)

We finish the dashboard by wiring our Model into both the View and the Controller:

###########################
# Data Manipulation / Model
###########################

def fetch_data(q):
    result = pd.read_sql(
        sql=q,
        con=conn
    )
    return result


def get_divisions():
    '''Returns the list of divisions that are stored in the database'''

    division_query = (
        f'''
        SELECT DISTINCT division
        FROM results
        '''
    )
    divisions = fetch_data(division_query)
    divisions = list(divisions['division'].sort_values(ascending=True))
    return divisions


def get_seasons(division):
    '''Returns the seasons of the datbase store'''

    seasons_query = (
        f'''
        SELECT DISTINCT season
        FROM results
        WHERE division='{division}'
        '''
    )
    seasons = fetch_data(seasons_query)
    seasons = list(seasons['season'].sort_values(ascending=False))
    return seasons


def get_teams(division, season):
    '''Returns all teams playing in the division in the season'''

    teams_query = (
        f'''
        SELECT DISTINCT team
        FROM results
        WHERE division='{division}'
        AND season='{season}'
        '''
    )
    teams = fetch_data(teams_query)
    teams = list(teams['team'].sort_values(ascending=True))
    return teams


def get_match_results(division, season, team):
    '''Returns match results for the selected prompts'''

    results_query = (
        f'''
        SELECT date, team, opponent, goals, goals_opp, result, points
        FROM results
        WHERE division='{division}'
        AND season='{season}'
        AND team='{team}'
        ORDER BY date ASC
        '''
    )
    match_results = fetch_data(results_query)
    return match_results


def calculate_season_summary(results):
    record = results.groupby(by=['result'])['team'].count()
    summary = pd.DataFrame(
        data={
            'W': record['W'],
            'L': record['L'],
            'D': record['D'],
            'Points': results['points'].sum()
        },
        columns=['W', 'D', 'L', 'Points'],
        index=results['team'].unique(),
    )
    return summary


def draw_season_points_graph(results):
    dates = results['date']
    points = results['points'].cumsum()

    figure = go.Figure(
        data=[
            go.Scatter(x=dates, y=points, mode='lines+markers')
        ],
        layout=go.Layout(
            title='Points Accumulation',
            showlegend=False
        )
    )

    return figure
Notes:

Run Application

Let's run app.py to make sure everything works.

$ export DB_URI=sqlite:///soccer-stats.db
$ python app.py
* Running on http://0.0.0.0:8050/ (Press CTRL+C to quit)
* Restarting with stat

Open a web browser...
Soccer Results Dashboard

And we're good to go!


Conclusion

Dash is a Python library that simplifies data-driven web app development. It combines Python's powerful data ecosystem with one of JavaScript's most popular front-end libraries (React).

In a future post, I will walk through the process of converting a React component from npm into a Dash-useable component. Stay tuned.


Additional Resources


 
    
 
 

Comments