The Edge Python API#

EdgeSession#

class edge.api.EdgeSession#

EdgeSession objects serve as your "entry point" into the Edge API. Currently there is no public way to construct these objects; you should always use the automatically-generated edge object present in new Jupyter notebooks.

To access the Edge files API, use the EdgeSession.files property.

property files: Folder#: Provides access to the Edge file system for your organization. The folder it returns is the "root folder", i.e. the base of the file tree. You can open files and other folders as needed from here.

property sources: EdgeData#: Provides access to Data Connectors in Edge, for your organization. You can use this interface to access and download remote data.

EdgeSession.search(query, kind=None)#

Search the object collections

Parameters

query (str) -- String to search for (uses trigram, so can be approximate)
kind (str) -- Search limiting filter, one of 'file', 'folder', 'connector'

Returns

List of search results.

Return type

list(SearchResult)

class edge.api.SearchResult#

Represents a result from EdgeSession.search().

property SearchResult.name#: Name of the matching object (i.e. the file basename)

property SearchResult.kind#: Kind of object (e.g. 'file' or 'folder')

property SearchResult.uid#: Universally-unique identifier for the object

Files API#

class edge.api.Folder#

Represents a folder in the Edge file system.

Folder.list()#

List all file and folder names in this folder.

Returns: List of File and Folder names.
Return type: list(tr)
Raises: NotFound --

Folder.list_files()#

List all file names in this folder.

Returns: List of file names.
Return type: list(str)
Raises: NotFound --

Folder.list_folders()#

List all folder names in this folder.

Returns: List of folder names.
Return type: list(str)
Raises: NotFound --

Folder.open(name, open_with=None)#

Open an existing file or folder.

Parameters

name (str) -- The name of the file or folder to open. May be a delimited series of folder names like "/this/folder/file" and we will walk all folders and open the final one.
open_with (str) -- Optional; name of an opener to use to open the object. Examples are "pandas" or "imageio", or "file" to get a file-like object giving access to the raw bytes. If not specified, an opener will be automatically picked for you.

Return type

object, File, or Folder

Raises

NotFound --
ValueError --

Folder.download(name, localpath=None)#

Download to a local file.

Currently, only files are supported.

Parameters

name (str) -- Path/name of the remote object
localpath (str, optional) -- Download to this path locally. Defaults to the same basename in the current working directory.

Folder.upload(destination, fp, overwrite=False)#

Upload a file to a folder.

Parameters

destination (str) -- The path/name of the file upload destination.
fp (file-like) -- An object having callable read() returning bytes object.
overwrite (bool) -- If true, any existing file will be overwritten

Raises

NotFound --

Folder.make_folder(name)#

Make a new folder.

Parameters: name (str) -- The name of the folder to create under this folder. May be a delimited series of folder names like "/this/folder/path" and we will ensure all folders in that series are created.
Return type: Folder

Folder.move(source, destination)#

Move a file or folder.

Parameters

source (str) -- The file or folder in this folder to move, or path to source
destination (str) -- The path to the file to move to.

Folder.rename(source, name)#

Move a file or folder.

Parameters

source (str) -- The file or folder in this folder to rename.
name (str) -- The new name for the source object

Folder.delete(object)#

Delete a file or folder.

Parameters: object (str) -- The file or folder to delete, or path to object to delete

class edge.api.File#

Represents a file in the Edge file system. You can get one of these via Folder.open(filename). Please note files are read-only; to change a file's contents, upload (and overwrite) using Folder.upload.

This class also has all file-like methods defined on Python's io.RawIOBase interface, appropriate to a read-only file.

property File.basename#: Name of the File object.

property File.size#: Size of the File object

File.read(size=-1)#

File.write()#

File.seek(offset, whence=0)#

Change stream position.

Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:

0 -- start of stream (the default); offset should be zero or positive
1 -- current stream position; offset may be negative
2 -- end of stream; offset is usually negative

Return the new absolute position.

File.tell()#: Return current stream position.

File.download_url()#

Data Sources API#

class edge.api.EdgeData#

This object is available at EdgeSession.sources.

EdgeData.list()#

List names of stored external connections.

Returns: Names of stored external connections.
Return type: list(str)

EdgeData.get(connection_id)#

Retrieve a stored connection object, given its name or UID.

Parameters

connection_id (str) -- Name or UID of the connection to retrieve.

Returns

The object corresponding to the retrieved connection.

Return type

IConnection

Raises

NotFound -- Raised if a connection with the given name is not found.
BadEntry -- If the connector definition or name is invalid.

EdgeData.add(connection)#

Adds a connection to the database

Parameters: connection (IConnection) -- A connector to add to the database.

EdgeData.update(connection)#

Updates a connection in the database

Parameters: connection (IConnection) -- A connector to update in the database.

EdgeData.remove(connection_id)#

Delete a given connection from storage.

Parameters: connection_id (str) -- Name or UID of the connection to remove.

SQL Data Connector#

class edge.api.SQLConnection(name, title, host, port, db_kind, db_name, username, password)#

Represents a connection to a remote SQL data store.

You can access table contents via the .tables dictionary-like attribute.

Examples for use:

# List table names
[1]: list(conn.tables)
['experiments', 'materials']

# Retrieve a table proxy object
[2]: mytable = conn.tables['experiments']

# Fetch the table contents as a Pandas dataframe
[3]: df = mytable.to_dataframe()

property kind#: Connection kind, for discrimination

property uid#: Connection unique ID

property name#: Connection name, for internal use

property title#: Title for display

property engine#: SQL Engine

property tables#: SQL Tables

test(timeout=None)#

Test the connection to the external database

Parameters: timeout (int, optional) -- The time in seconds to wait before reporting a connection as failed. If not specified, it defaults to a short timeout.
Return type: ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS, or NO_CONNECTION)

to_dict()#

Translates the connection as a dictionary, for serialization.

Returns: Connection contents in dictionary form.
Return type: Dict

class edge.api.SQLTable(engine_proxy, name, *args, **kwargs)#

Container for data from an external SQL Table.

to_dataframe()#

Retrieve a dataframe from the underlying tabular data.

Returns: A dataframe containing the tabular data.
Return type: pandas.DataFrame

S3 Data Connector#

class edge.api.S3Connection(name, title, bucket, endpoint, access_key, secret_key, _verify=None)#

Represents a connection to a remote S3 bucket.

Use the conn.root attribute to access the contents as a file system.

Examples for use:

# List files and folders
[1]: conn.root.list()
['myfile.jpg', 'mysubfolder']

# Open a file
[2]: img = conn.root.open('myfile.jpg')

# Open a "folder" in S3
[3]: myfolder = conn.root.open('myfolder')

# List files in that subfolder
[4]: myfolder.list()
['example.jpg']

# Download a file
[5]: myfolder.download('example.jpg', localpath='example_downloaded.jpg')

property kind#: Connection kind, for discrimination

property uid#: Connection unique ID

property name#: Connection name, for internal use

property title#: Title for display

property bucket#: Underlying S3 bucket

property root#: Access an S3 bucket as a file system

iter_prefixes(prefix=None)#

Search for key prefixes in the bucket.

Key prefixes can be used to organise the keys in a bucket in a structure that resembles a folder. The result contains strings between an optional prefix and and the first occurrence of the delimiter character, which is assumed to be "/". For example, if the bucket contains two keys named "folder1/myfile.txt" and "folder2/subfolder/myfile2.txt", iter_prefixes() finds the keys "folder1/" and "folder2/", while iter_prefixes(prefix='folder2/') finds "folder2/subfolder/".

Parameters: prefix (str, optional) -- Base prefix to use for searching other prefixes. Defaults to an empty prefix.
Returns: An iterator yielding the prefixes found.
Return type: iterator(str)

list_prefixes(prefix=None)#

Search for key prefixes in the bucket.

Parameters: prefix (str, optional) -- Base prefix to use for searching other prefixes. Defaults to an empty prefix.
Returns: A list containing the prefixes found.
Return type: list(str)

iter_keys(prefix=None)#

Search for keys in the bucket.

Parameters: prefix (str, optional) -- Limit the search to keys starting with a given prefix.
Returns: A list of the keys found in the bucket.
Return type: iterator(str)

list_keys(prefix=None)#

Search for keys in the bucket.

Parameters: prefix (str, optional) -- Limit the search to keys starting with a given prefix.
Returns: A list of the keys found in the bucket.
Return type: list(str)

test(timeout=None)#

Test the connection to the external filestore

Parameters: timeout (int, optional) -- The time in seconds to wait before reporting a connection as failed. If not specified, it defaults to a short timeout.
Return type: ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS, or NO_CONNECTION)

open(key, open_with=None)#

Retrieve data from an S3 object as an appropriate Python object.

Parameters

key (str) -- The S3 bucket key from which to retrieve data
open_with (str, optional) -- Opener to use, for example "pandas" or "imageio". Using "file" will give a file-like object with access to the raw bytes. If not specified, an appropriate opener will be selected for you.

Returns

The most appropriate type found for the underlying data. If no specific type is found, a generic File is returned.

Return type

Any

Raises

ValueError -- Raised if requested type is not known.
NotFoundError -- Raised when the given key is not found in the bucket.

to_dict()#

Translates the connection as a dictionary, for serialization.

Returns: Connection contents in dictionary form.
Return type: Dict

class edge.api.S3Folder(_conn, _prefix)#

Represents a "folder" in an S3 bucket.

You can use this object to explore the contents of an S3 bucket interactively. S3 prefixes are mapped to folders, S3 keys are mapped to files.

[1]: s3conn.root.list_folders() ['folder1', 'folder2']

[2]: folder = s3conn.root.open('folder1') [3]: folder.list_files() ['myimg.jpg']

[4]: img = folder.open('myimg.jpg')

Private constructor

list_folders()#

List all folder names in this folder.

Returns: List of folder names.
Return type: list(str)

list_files(show_all=False)#

List all file names in this folder.

Returns

list(str) -- List of file names.
show_all(bool) -- Also show files hidden by default. Currently this means file names which end with the string $folder$ .

list(show_all=False)#

List all file and folder names in this folder.

Returns

list(tr) -- List of File and Folder names.
show_all(bool) -- Also show files hidden by default. Currently this means file names which end with the string $folder$ .

open(name, open_with=None)#

Retrieve an S3 object as an appropriate Python object.

Parameters

name (str) -- The name of the file or folder to open.
open_with (str, optional) -- Opener to use, for example "pandas" or "imageio". Using "file" will give a file-like object with access to the raw bytes. If not specified, an appropriate opener will be selected for you.

Returns

The most appropriate type found for the underlying data. If no specific type is found, a generic File is returned.

Return type

Any

Raises

ValueError -- Raised if requested type is not known.
NotFoundError -- Raised when the given key is not found in the bucket.

download(name, localpath=None)#

Download to a local file.

Currently, only files are supported.

Parameters

name (str) -- Name of the remote object
localpath (str, optional) -- Download to this path locally. Defaults to the same basename in the current working directory.

class edge.api.S3File(bucket_object, context=None)#

File-like providing remote access to an S3 object for S3Connector.

Construct a new File object.

Parameters

bucket_object (S3.Object) -- The object in a bucket this file represents
context (SSLContext) -- Optional SSL Context for S3 accesses - used in testing to disable SSL. Do not specify unless you have a strong need.

property basename#: Name of the File object.

property size#: Size of the File object

readable()#

Return whether object was opened for reading.

If False, read() will raise OSError.

seekable()#

Return whether object supports random access.

If False, seek(), tell() and truncate() will raise OSError. This method may need to do a test seek().

writable()#

Return whether object was opened for writing.

If False, write() will raise OSError.

tell()#: Return current stream position.

seek(offset, whence=0)#

Change stream position.

Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:

0 -- start of stream (the default); offset should be zero or positive
1 -- current stream position; offset may be negative
2 -- end of stream; offset is usually negative

Return the new absolute position.

OpenAPI Data Connector#

class edge.api.OpenAPIConnection(name, title, host, specs_url)#

A connector generating a REStful API from OpenAPI 3 specifications.

Example of use:

# Create a connector
[1]: conn = OpenAPIConnector('petstore', 'Petstore Demo',
...     host="http://petstore3.swagger.io",
...     specs_url="https://petstore3.swagger.io/api/v3/openapi.json")

# Generate a client allowing API access.
# This may take a few seconds as it needs to generate source code.
[2]: client = conn.generate_new_client()

# Use the client to query the API
[3]: pets = client.pet.findPetsByStatus()

Create a new OpenAPI3 connector.

Parameters

name (Str) -- Connection name, must be unique
title (Str) -- Title as displayed in the Data App
host (Str) -- URL for the API host, including the http/https scheme
specs_url (Str) -- URL for the OpenAPI3 specification, including the http/https scheme.

property kind#: Connection kind, for discrimination

property uid#: Connection unique ID

property name#: Connection name, for internal use

property title#: Title for display

property host#: Host for which we are generating an API

property specs_url#: URL which points to the OpenAPI3 JSON spec file.

to_dict()#

Translates the connection as a dictionary, for serialization.

Returns: Connection contents in dictionary form.
Return type: Dict

test()#

Test the connection to the external data source

Note that the timeout should default to a short value, so a connection failure could also mean the network is slow or lagging.

Parameters

timeout (int) -- The time in seconds to wait before reporting a connection as failed.

Returns

ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS,
or NO_CONNECTION)

generate_new_client()#

Get a client allowing access to the remote API.

The remote API is available via methods on the returned object and its attributes. You can explore it by using e.g. IPython completion, or by using dir().

This method will generate Python files with the API binding, and cache them on disk. This is a fairly expensive process, so once a client is generated, you should try to use it for as long as possible. All files are cleaned up with the object becomes unreachable (or Python exits).