/ Data Visualization

Dynamic web based reports/dashboards in Python

Dynamic web based reports/dashboards in Python

awe

Dynamic web based reports/dashboards in Python.

What is awe for?

awe use cases:

  • Create a report for some data you collected in your scripts.
  • Poll some data in your script and update a chart with it.
  • A replacement for print statements in your scripts that can include
    interactive tables, charts, headers, colors, etc... with minimum fuss.

awe isn't for you if you need to:

  • Handle large amounts of data. awe is quite wasteful in terms of resources. This works
    well for small-ish amounts of data. On the other hand, charts with many points will
    probably make your browser completely unresponsive.

Under the hood, awe generates the page using react.

Installation

pip install awe

Getting Started

Begin by creating an awe.Page() instance. e.g:

from awe import Page
page = Page()

A page is built by creating a hierarchy of elements.

Every element, including the root Page element, exposes new_XXX() methods that create element children.

These methods can create leaf elements such as new_text(), new_table(), etc... e.g:

page.new_text('Hello there')

They can also create container elements such as new_tabs(), new_card() etc... e.g:

card = page.new_card()

If you don't intend to dynamically add data to an element, you can simply call the new_XXX() method with appropriate
arguments and be done with it.

If you do plan on adding data dynamically or create some element hierarchy, then keep a reference to the created
element, returned by the new_XXX() call. e.g:

card = page.new_card()
text = card.new_text('Text inside of card')
button = card.new_button(lambda: None)

The above creates a card as a child element of page and text and button elements as children of card.

Once you're done with the initial page setup, call page.start(). e.g:

# The default call will open a browser page without blocking the script
page.start()

# This will block the script
page.start(block=True)

# This will prevent the default browser open behavior
page.start(open_browser=False)

The examples section can be used as reference for the different elements that can be created with awe.

Examples

hello_world.py (static demo)

The most basic page with a single text element.

from awe import Page


def main():
    page = Page()
    page.new_text('Hello World!')
    page.start(block=True)


if __name__ == '__main__':
    main()

hello_world

chart_simple.py (static demo)

A page with a single chart.

The chart is initialized with a single data item and then updated every 1 second with a new data item.

The data added to the chart is transformed by the numbers transformer. It builds a single chart with series
built from each value in the data item (which is a list of numbers)

import time
from random import randint

from awe import Page


def generate_random_data(size, num_series):
    result = []
    for _ in range(size):
        item = []
        for i in range(num_series):
            item.append(randint(i*100, i*100 + 100))
        result.append(item)
    return result


def main():
    args = (1, 3)
    page = Page()
    data = generate_random_data(*args)
    chart = page.new_chart(data=data, transform='numbers')
    page.start()
    while True:
        time.sleep(1)
        chart.add(generate_random_data(*args))


if __name__ == '__main__':
    main()

chart_simple

chart_complex.py (static demo)

A page with a single chart.

The chart is initialized with a single data item and then updated every 5 seconds.

The chart has a moving time window of 3 minutes.

The data added to the chart is transformed by the 2to31 transformer. It builds charts from the different keys
of the 2nd level in the nested dictionary data items. It builds the chart series from the different combinations
of the 3rd and 1st levels in the nested dictionary data items.

In general, every [Ns...]to[Ms...] transformer is supported.

import time
from random import randint

from awe import Page


def generate_random_data(size):
    level3 = lambda: {'l3_key1': randint(0, 1000), 'l3_key2': randint(0, 1000)}
    level2 = lambda: {'l2_key1': level3(), 'l2_key2': level3(), 'l2_key3': level3()}
    level1 = lambda: {'l1_key1': level2(), 'l1_key2': level2()}
    return [level1() for _ in range(size)]


def main():
    page = Page()
    data = generate_random_data(1)
    chart = page.new_chart(data=data, transform='2to31', moving_window=3 * 60)
    page.start()
    while True:
        time.sleep(5)
        chart.add(generate_random_data(1))


if __name__ == '__main__':
    main()

chart_complex

chart_flat.py (static demo)

A page with a single chart.

The chart is initialized with a single data item and then updated every 0.7 seconds with a new data item.

The chart has a moving time window of 3 minutes.

The data added to the chart is transformed by the flat transformer. It builds charts from the different combinations
of the chart_mapping list. It builds the chart series from the different combinations
of the series_mapping list. The values are extracted from the value_key key.

import time
import random

from awe import Page


def generate_random_data():
    return [{
        'color': random.choice(['blue', 'yellow']),
        'fruit': random.choice(['apple', 'orange']),
        'temp': random.choice(['cold', 'hot']),
        'city': random.choice(['Tel Aviv', 'New York']),
        'value': random.randint(1, 100)
    }]


def main():
    page = Page()
    data = generate_random_data()
    chart = page.new_chart(data=data, transform={
        'type': 'flat',
        'chart_mapping': ['color', 'fruit'],
        'series_mapping': ['temp', 'city'],
        'value_key': 'value'
    }, moving_window=3 * 60)
    page.start()
    while True:
        time.sleep(0.7)
        chart.add(generate_random_data())


if __name__ == '__main__':
    main()

chart_flat

page_properties.py (static demo)

A page that demonstrates how to set the page title, width and override its style.

from awe import Page


def main():
    page = Page('Page Properties', width=600, style={
        'backgroundColor': 'red'
    })
    page.new_card('hello')
    page.start(block=True)


if __name__ == '__main__':
    main()

page_properties

button_and_input.py (static demo)

A page with a button and two inputs.

Clicking the button or hitting enter when the second input is focused, runs do_stuff
which gets a reference to the input values and the button element using the @inject decorator.

do_stuff in turn, updates the button text.

from awe import Page, inject


@inject(variables=['input1', 'input2'], elements=['button1'])
def do_stuff(input1, input2, button1):
    text = '{} {} {}'.format(button1.count, input1, input2)
    button1.text = text
    button1.count += 1


def main():
    page = Page()
    b = page.new_button(do_stuff, id='button1')
    b.count = 0
    page.new_input(id='input1')
    page.new_input(
        placeholder='Input 2, write anything!',
        on_enter=do_stuff,
        id='input2'
    )
    page.start(block=True)


if __name__ == '__main__':
    main()

button_and_input

standard_output.py (static demo)

A page that demonstrates adding text dynamically to a page after it has been started.

The elements are created with a custom style.

import time

from awe import Page


def main():
    page = Page()
    page.start()
    page.new_text('Header', style={'fontSize': '1.5em', 'color': '#ff0000'})
    for i in range(20):
        page.new_text('{} hello {}'.format(i, i), style={'color': 'blue'})
        time.sleep(2)
    page.block()


if __name__ == '__main__':
    main()

standard_output

collapse.py (static demo)

A page with a single collapse element.

The collapse has 3 panels. The first panel defaults to being expanded. The other two panels default to collapsed.

from awe import Page


def main():
    page = Page()
    collapse = page.new_collapse()
    panel1 = collapse.new_panel('Panel 1', active=True)
    panel1.new_text('Hello From Panel 1')
    panel2 = collapse.new_panel('Panel 2', active=False)
    panel2.new_text('Hello From Panel2')
    panel3 = collapse.new_panel('Panel 3')
    panel3.new_text('Hello From Panel3')
    page.start(block=True)


if __name__ == '__main__':
    main()

collapse

showcase.py (static demo)

A page that showcases all (currently) available elements in awe.

import time

from awe import Page


def main():
    now = time.time()
    page = Page('Showcase')
    grid = page.new_grid(columns=3, props={'gutter': 12})
    grid.new_card('Card 1')
    card = grid.new_card()
    tabs = grid.new_tabs()
    collapse = grid.new_collapse()
    grid.new_chart([(now+i, -i) for i in range(100)], transform='numbers')
    grid.new_table(['Header 1', 'Header 2', 'Header 3'], page_size=4).extend([
        ['Value {}'.format(i), 'Value {}'.format(i+1), 'Value {}'.format(i+2)]
        for i in range(1, 20, 3)
    ])
    grid.new_divider()
    grid.new_button(lambda: None, 'Button 1', block=True)
    grid.new_input()
    grid.new_icon('heart', theme='twoTone', two_tone_color='red')
    inline = grid.new_inline()
    card.new_text('Card Text 1')
    card.new_text('Card Text 2')
    tabs.new_tab('Tab 1').new_text('Tab 1 Text')
    tabs.new_tab('Tab 2').new_text('Tab 2 Text')
    tabs.new_tab('Tab 3').new_text('Tab 3 Text')
    tabs.new_tab('Tab 4').new_text('Tab 4 Text')
    collapse.new_panel('Panel 1', active=True).new_text('Panel 1 Text')
    collapse.new_panel('Panel 2').new_text('Panel 2 Text')
    collapse.new_panel('Panel 3').new_text('Panel 3 Text')
    collapse.new_panel('Panel 4').new_text('Panel 4 Text')
    collapse.new_panel('Panel 5').new_text('Panel 5 Text')
    inline.new_inline('inline 1')
    inline.new_inline('inline 2')
    inline.new_inline('inline 3')
    page.start(True)


if __name__ == '__main__':
    main()

showcase

kitchen.py (static demo)

A page that showcases many different element types supported by awe.

The following element types are used:

  • tabs
  • grids
  • dividers
  • cards
  • texts
  • tables

Element data is updated using API exposed by each element type.

In addition, the divider element is updated using the lower level element.update_prop() method which updates
the underlying props of the react component.

import time

from awe import Page


class Kitchen(object):
    def __init__(self, parent):
        self.tabs = parent.new_tabs()
        self.tab1 = Tab1(self.tabs)
        self.tab2 = Tab2(self.tabs)

    def update(self, i):
        self.tab1.update(i)
        self.tab2.update(i)


class Tab1(object):
    def __init__(self, parent):
        self.tab = parent.new_tab('Tab 1')
        self.grid = Grid(self.tab)
        self.divider = self.tab.new_divider()
        self.divider2 = self.tab.new_divider()
        self.table2 = self.tab.new_table(headers=['c 4', 'c 5'], page_size=5)

    def update(self, i):
        self.divider.update_prop('dashed', not self.divider.props.get('dashed'))
        self.table2.prepend([-i, -i * 12])
        self.grid.update(i)


class Grid(object):
    def __init__(self, parent):
        self.grid = parent.new_grid(columns=3)
        self.table1 = self.grid.new_table(headers=['c 1', 'c 2', 'c 3'], cols=2, page_size=5)
        self.cc = self.grid.new_card()
        self.cc_inner = self.cc.new_card('inner')
        self.ct = self.grid.new_card()
        self.t1 = self.ct.new_text('4 Text')
        self.t2 = self.ct.new_text('4 Text 2')
        self.card = self.grid.new_card('0 Time')
        self.card2 = self.grid.new_card('6')
        self.card3 = self.grid.new_card('7', cols=3)

    def update(self, i):
        self.table1.append([i, i ** 2, i ** 3])
        self.card.text = '{} Time: {}'.format(i, time.time())
        self.t1.text = '4 Text: {}'.format(i * 3)
        self.t2.text = '4 Text {}'.format(i * 4)


class Tab2(object):
    def __init__(self, parent):
        self.tab = parent.new_tab('Tab 2')
        self.table3 = self.tab.new_table(headers=['c 6', 'c 7', 'c 8'], page_size=5)
        self.table4 = self.tab.new_table(headers=['c 2', 'c 5'], page_size=5)

    def update(self, i):
        self.table3.append([-i, -i ** 2, -i ** 3])
        self.table4.append([i, i * 12])


def main():
    page = Page()
    kitchen = Kitchen(page)
    page.start()
    try:
        for i in range(1000):
            kitchen.update(i)
            time.sleep(5)
    except KeyboardInterrupt:
        pass


if __name__ == '__main__':
    main()

kitchen

Supported Python Versions

Tested on Python 2.7.15 and 3.7.1

Should work on many earlier versions I suppose, but haven't been tested so you can't be sure.

These days, I'm mostly working with Python 2.7, so things may unintentionally break on Python 3.
That being said, the test suite runs on both versions, so chances of that happening are not very high.

Support for Python 3 has been added after initial development, so please open an issue if something
seems broken under Python 3. In fact, open an issue if something seems broken under any Python version :)

Export To Static HTML

At any point during the lifetime of a page you can export its current state to a standalone html file you can
freely share.

You can export in any of the following ways:

  • Open the options by clicking the options button at the top right and then click Export.
  • Open the options by holding Shift and typing A A A (three consecutive A's) and then click Export.
  • Hold Shift and type A A E (two A's then E).

Note that for the keyboard shortcuts to work, the focus should be on some page content.

Export function

By default, when you export a page, the result is simply downloaded as a static file.

You can override this default behavior by passing an export_fn argument when creating the Page instance. e.g:

import time

from awe import Page

from utils import save_to_s3  # example import, not something awe comes bundled with


def custom_export_fn(index_html):
    # index_html is the static html content as a string.
    # You can, for example, save the content to S3.
    key = 'page-{}.html'.format(time.time()) 
    save_to_s3(
        bucket='my_bucket', 
        key=key, 
        content=index_html
    )
    
    # Returning a dict from the export_fn function tells awe to skip the default download behavior.
    # awe will also display a simple key/value table modal built from the dict result.
    # Returning anything else is expected to be a string that will be downloaded in the browser.
    # This can be the unmodified index_html, a modified one, a json with statistics, etc...
    return {'status': 'success', 'key': key}


def main():
    page = Page(export_fn=custom_export_fn)
    page.new_text('Hello')
    page.start(block=True)


if __name__ == '__main__':
    main()

Offline

You can also generate the page content offline, in python only and export it in code by calling page.export().

The return value of export is the return value of export_fn which defaults to the static html content as string.

e.g:

from awe import Page

def main():
    page = Page(offline=True)
    page.new_text('Hello')
    print page.export()
    # you can override the export_fn supplied during creation by passing
    print page.export(export_fn=lambda index_html: index_html[:100])
    

if __name__ == '__main__':
    main()

Why is it named awe?

I like short names. I initially called this package pages but then discovered it is already taken in pypi.
Finding a decent unused name is not an easy task!

GitHub