Software architecture

Overview

The Ichnosat goal is to Download satellite data and Process this data automatically. The high level workflow of the platform is simple:


_images/overview.png



_images/ichnosat-workflow.png


Downloader and Processor are the main modules of this platform. The schema with all modules involved follows:



_images/ichnosat-modules.png


Downloader

This module is deputed to retrieve the list of available products and download them, on the basis of the configurations set by user. The possible configurations to filter the list of products are:

  • List of tiles
  • List of files to download for each tile
  • Sensing time interval (start - end )

Class diagram



_images/downloader-class-diagram.png


Sequence diagram



_images/downloader-sequence-diagram.png


Processor

The goal of this component is to process downloaded products. Every processing algorithm is represented by C++ plugins. This means that Processor module is extendible. Plugins are dynamic shared library (Linux environment)

The processor runs as http server to receive requests via http:



_images/processor-web-interface.png


When the POST /process http is received the Processor starts a processing task, spreading a pool of threads. The number of threads is configurable via conf.cfg file.

If you want to develop a new plugin for your processing purposes, please follows the tutorial:

Class diagram



_images/processor-class-diagram.png


Sequence diagram



_images/processor-sequence-diagram.png


System Manager

System Manager is the high level module to manage other modules.

Class diagram



_images/system_manager-class-diagram.png


Database

Database module contains the classes of ORM. In this version (0.1) is used SqlAlchemy ORM.

Class diagram



_images/database-class-diagram.png


Database schema



_images/database-schema.png


Folder Structure

The description of the main folders in the source code follows:

|-- Dockerfile             # Docker file of ichnosat platform
|-- LICENSE.TXT            # MIT license text
|-- README.md              # README of the project (for github main page)
|-- auto-docs              # Documentation sources and images
|-- data_local
|   |-- db                 # Postgresql database files (mounted by docker-compose)
|   |-- inbox              # Temporary downloaded products by the Downloader
|   |-- log                # Ichnosat logs
|   |-- outbox             # Processed products folder
|   |-- supervisord        # Supervisord log folder
|-- docker-compose.yml     # Docker Compose definition file
|-- src
|   |-- core               # Main Modules source code (Downloader, Processor, System Manager)
|   |-- data               # Database and Logger source code
|   |-- gui                # Web GUI interface source code
|   |-- presentation       # External interface (http) source code
|   `-- tests              # Tests
`-- vendors                # Third part dependencies

External Inteface APIs



_images/external-interface-apis.png


Technologies

  • Docker and docker compose: virtual image container
  • SqlAlchemy: ORM framework for python
  • Supervisord: Task manager
  • Valgrind: Process monitoring
  • Sphinx: Automatic Documentation for python
  • Flask: light webserver
  • Nginx: web server
  • Postgresql: database
  • GDAL: C/C++ library for satellite image processing
  • OpenJPG: library to manage jpeg2000 images
  • Python 3.4
  • crontab
  • debian
  • pip
  • C++11

Code documentation

For more details about methods internals, please visit the section of Code APIs documentation: