Architecture documentation

This section of the documentation contains architecture docs, following the C4 documentation model.

System Context

The system context shows the relationships between the system that Enthought is building and the external system components that it interacts with. It also briefly details the relationships between users and developers associated with the project.

Notice that there are really two layers of system context here: one boundary separates the customer from Enthought-internal projects, and another separates the Edge team from other Enthought teams.

@startuml System Context Diagram
!include C4-PlantUML/C4_Context.puml
!include C4-PlantUML/C4_Container.puml

' Scenario:

title System Context diagram for Edge

LAYOUT_WITH_LEGEND()
LAYOUT_TOP_DOWN()

' Sub-personas for this are split out into lower level diagrams
Person(researcher, "Researcher", "Domain expert, performing analysis and making decisions")
Person(ext_it, "Customer I.T", "I.T Department personnel within the customer organistion")
' Enthought includes teams outside the edge team, like devops
' customer-facing DTX team, ...
Boundary(enthought, "Enthought") {
    Person(devops, "Dev Ops", "The Enthought DevOps team")
    Person(dtx_team, "DTX team", "The consulting team working with the customer")
    Container(\
        customer_app, "Customer App", "Python; Other technologies",\
        "A general application specified by a customer",\
    )
    ContainerDb(data_catalog, "Data Catalog Service", "Web Service; SQL, Python")


    ' Inner boundary separates containers that are the responsibility of the
    ' Edge team from those that are the responsibility of other Enthoughters
    ' or outside the Enthought system entirely.
    System_Boundary(enthought_dm_doe, "Enthought Edge") {
        Container(\
            edge_app, "Edge", "GUI application; Python,\
            Canopy Data", "Allows users to browse, search, and manage data"\
        )
        Person(product_dev_team, "Product Development Team")
    }
}

Rel_L(researcher, edge_app, "manages data using", "GUI application; Python")
Rel(researcher, dtx_team, "specifies feature requirements to")
Rel_L(dtx_team, researcher, "provides training & support for application")
Rel(dtx_team, edge_app, "Extends data management application via entrypoints")
Rel(dtx_team, devops, "Specifies customer data requirements to")
Rel_D(devops, data_catalog, "Sets up service")
Rel(edge_app, data_catalog, "Stores/Retrieves data", "HTTP")
Rel(dtx_team, customer_app, "Builds and maintains")
BiRel_R(ext_it, devops, "Coordinate to set up Data Catalog infrastructure")
Rel(customer_app, edge_app, "Makes use of library code from", "Python")
Rel(dtx_team, product_dev_team, "Relays feature requirements to")
Rel_R(product_dev_team, edge_app, "Implements feature requests; Maintains codebase")
@enduml

Containers

The container level expands the "Edge" system from the previous diagram and fills out the separate parts making up the system. While items at the container level are generally assumed to be separately deployable, here we split out the API into its own container, due to the fact that it can also be used meaningfully outside the context of a GUI application.

@startuml Container diagram for Edge
!include C4-PlantUML/C4_Context.puml
!include C4-PlantUML/C4_Container.puml


title Container diagram for Edge

LAYOUT_WITH_LEGEND()

Person(researcher, "Researcher", "Domain expert, performing analysis and making decisions")
' People interacting with the system outside the context of a GUI application
Person(data_scientist, "Data Scientist", "Creates scripting workflows using domain specific data")

Boundary(enthought, "Enthought"){
    Person(enthought_dev, "DTX Team", "The consulting team working with the customer")
    Person(enthought_dev_alt, "DTX Team", "The consulting team working with the customer")
    System_Boundary(ms_dm_doe, "Edge"){
        Container(\
            edge_app, "Edge Data Management Application",\
            "GUI Application; Python, Canopy Data",\
            "Application for data and data model management"\
        )
        Container(\
            ms_doe_app, "MS DOE Application",\
            "GUI Application; Python",\
            "Application providing guided experimental design features"\
        )
        Container(\
            edge_lib, "Edge Library",\
            "Python Package; Python",\
            "Library for data and data model management"\
        )
    }
    ContainerDb(\
        data_catalog, "Data Catalog Service", "Web Service; SQL, Python",\
        "Database containing scientific data"\
    )
}
Rel_D(edge_app, edge_lib, "Makes API calls using", "Python")
Rel_D(ms_doe_app, edge_lib, "Makes API calls using", "Python")
Rel(edge_lib, data_catalog, "Reads from and writes to", "HTTP")
Rel_R(data_scientist, edge_lib, "Writes scripts or programs which use", "Python")
Rel(researcher, edge_app, "Manages data using", "GUI Application")
Rel_R(researcher, ms_doe_app, "Generates experimental parameters", "GUI Application")
Rel_U(enthought_dev, edge_app, "Extends Edge application via additional plugins")
Rel(enthought_dev, edge_lib, "Uses library in external applications", "Python; Other Technologies")
Rel_R(enthought_dev_alt, ms_doe_app, "Extends application via entrypoints for additional classes")
Rel_R(data_scientist, ms_doe_app, "Extends application via entrypoints for additional classes")
BiRel(data_scientist, enthought_dev_alt, "Collaborate on extension code")
@enduml

Components

The third level of the C4 architecture diagram gives some insight into the structure of the individual applications. Some personas from the upper levels are also split into more specialised roles. These roles are not mutually exclusive, for example someone who uses the Edge Application as a "Data Modeller" may also interact with the app as a "Researcher" or "Data Scientist".

@startuml Component Diagram for the Edge Library
!include C4-PlantUML/C4_Context.puml
!include C4-PlantUML/C4_Container.puml
!include C4-PlantUML/C4_Component.puml

title Component diagram for the Edge Library

LAYOUT_WITH_LEGEND()
Person(data_scientist, "Data Scientist", "Creates scripting workflows using domain specific data")
Boundary(edge_lib, "Edge Library"){
Component(edge_service, "Edge Service API", "python", "Provides functionality for data access")
Component(edge_scripting, "Edge Scripting API", "python", "Classes and methods for interacting with data model objects")
}
ContainerDb(\
        data_catalog, "Data Catalog Service", "Web Service; SQL, Python",\
        "Database containing scientific data"\
    )

Rel(data_scientist, edge_scripting, "Uses functions from")
Rel(edge_service, data_catalog, "Reads from and writes to", "HTTP")
Rel(edge_scripting, edge_service, "Makes API calls using")
@enduml

@startuml Component Diagram for Edge Application
!include C4-PlantUML/C4_Context.puml
!include C4-PlantUML/C4_Container.puml
!include C4-PlantUML/C4_Component.puml

title Component diagram for Edge Application

LAYOUT_WITH_LEGEND()

Person(researcher, "Researcher", "Domain expert, performing analysis and making decisions")
Person(data_modeller_persona, "Data Modeller", "Responsible for creating and updating data models for the overall project")
Person(data_engineer, "Data Importer", "Responsible for organising data importing practice for the overall project")
Boundary(ms_app_boundary, "Edge Application", "Canopy Data Application"){
    Component(data_searcher, "Data Searcher Plugin", "Python; Canopy Data Plugin", "Provides functionality for data searches")
    Component(data_modeller, "Data Modeller Plugin", "Python; Canopy Data Plugin", "Provides functionality for creating data models")
    Component(data_importer, "Data Importer Plugin", "Python; Canopy Data Plugin", "Provides functionality for importing data from spreadsheets/csv")
    Component(raw_data_import, "Raw Data Importer Plugin", "Python; Canopy Data Plugin", "Provides functionality for associating raw data files to an Entity in the Data Catalog")
}

Container(edge_lib, "Edge Library", "Python")
Container_Ext(endex, "Endex", "Python", "Library to facilitate the indexing, storage, retrieval, and parsing of files")
ContainerDb(\
    data_catalog, "Data Catalog Service", "Web Service; SQL, Python",\
    "Database containing scientific data"\
)
ContainerDb(\
    bulk_data_store, "Bulk Data Store", "Minio, S3",\
    "Data Storage for raw data files"\
)

Rel(data_modeller, edge_lib, "Sends and retrieves data models with")
Rel(data_searcher, edge_lib, "Sends search queries and retrieves data with")
Rel(data_importer, edge_lib, "Sends imported data with")
Rel(edge_lib, data_catalog, "Reads from and writes to", "HTTP")
Rel(raw_data_import, endex, "Uses functionality from")
Rel(endex, bulk_data_store, "Reads from and writes to", "HTTP")
Rel(data_modeller_persona, data_modeller, "Uses", "GUI Application")
Rel(researcher, data_importer, "Uses", "GUI Application")
Rel(researcher, data_searcher, "Uses", "GUI Application")
Rel(data_engineer, researcher, "Coordinates data importing practice")
Rel(researcher, raw_data_import, "Uses", "GUI Application")
@enduml

The final (code) level of the architecture is not represented in a diagram, as it is already covered by the design documentation for the project, along with the autogenerated API documentation.