imoog


A database-based CDN node supporting PostgreSQL and MongoDB backends.



Ubuntu host guide by Digital Ocean.

A simple to use database-based deployable CDN node for hobbyist developers who wish to have their own CDN!

Setup

  • Clone this repo via this command: git clone https://github.com/justanotherbyte/imoog
  • Go into the imoog/settings.py file and adjust your settings. Examples for both database drivers have been provided in the file.
  • Install a production asgi server of your choice. The 2 I recommend are hypercorn and uvicorn. Installing their base packages will suffice.
  • To automatically install the respective dependencies, please run pip install -r requirements.txt in the directory where you wish to store your node.
  • It is recommended to house the imoog folder within another folder, as there are other files that come with this repo, that are not housed within the imoog folder.

Running your ASGI server

Hypercorn

# inside the imoog directory
hypercorn app:app --graceful-timeout 3 --workers 3

# outside of the imoog directory
hypercorn imoog.app:app --graceful-timeout 3 --workers 3

Uvicorn

# inside the imoog directory
uvicorn app:app --workers 3 

# outside of the imoog directory
uvicorn imoog.app:app --workers 3

Please keep in mind that both Uvicorn and Hypercorn support running applications through the Unix Domain Socket (UDS) protocol rather than the Transmission Control Protocol (TCP). The choice is yours.

On another note, keep in mind that the amount of workers you have will only affect performance if your machine’s CPU core count supports it. Else, increasing the worker count will not be helpful at all.

To read more about sockets and which protocol will be the right one for you, please refer to this article: https://www.digitalocean.com/community/tutorials/understanding-sockets

Settings

  • imoog offers granular control over many key aspects of the node. Most of these can be extremely overwhelming. Go ahead and hop into the imoog/settings.py file, where you’ll find detailed explanations of each and every setting.
  • Another thing that can be overwhelming are the 2 database drivers. How do you configure them? What are their optimal settings. Again, everything is explained inside the imoog/settings.py file.

Proxy

  • In order to use this node cleanly, I recommend placing yourself behind a proxy server. One of the most popular choices is NGINX.

Caching + Cloudflare

  • The Imoog Node handles a lot of the caching for you, however, a good next step would be to place your CDN on cloudflare.
  • Cloudflare has some awesome caching solutions. Also, overall, they make it easy to expose your CDN to the open-web.

Uploading Files

This Node receives files via multipart/form-data. So it’s best if you were to adapt your upload system to this. The field name the node expects is just file, so please upload it with this name. Please remember to register the Authorization header. This key can be set in the imoog/settings.py file.

Upload example (aiohttp)

import aiohttp
import asyncio


async def main():
    session = aiohttp.ClientSession()
    form  = aiohttp.FormData()
    form.add_field("file", b'imagebyteshere', content_type="application/octet-stream")
    resp = await session.post("http://localhost:8000/upload", data=form, headers={"Authorization": "myawesomesecretkey"})
    returned_data = await resp.json()
    print(returned_data)
    await session.close()

asyncio.get_event_loop().run_until_complete(main())

>>> {'status': 200, 'file_id': 'FSTSH2RPI'}

Driver and dependency information

Internally, imoog uses multiple different libraries for the many different supported database and cache drivers.

  • PostgreSQL – asyncpg
  • MongoDB – motor
  • Redis – aioredis (You can optionally install aioredis[hiredis] for a C based Redis protocol parser for additional speeds)

All of these libraries are currently the best in their field for asynchronous client side connections to their respective databases.

GitHub

View Github