The Edge Python API#
EdgeSession#
- class edge.api.EdgeSession#
EdgeSession objects serve as your "entry point" into the Edge API. Currently there is no public way to construct these objects; you should always use the automatically-generated
edge
object present in new Jupyter notebooks.To access the Edge files API, use the
EdgeSession.files
property.- property files: Folder#
Provides access to the Edge file system for your organization. The folder it returns is the "root folder", i.e. the base of the file tree. You can open files and other folders as needed from here.
- property sources: EdgeData#
Provides access to Data Connectors in Edge, for your organization. You can use this interface to access and download remote data.
- EdgeSession.search(query, kind=None)#
Search the object collections
- Parameters
query (str) -- String to search for (uses trigram, so can be approximate)
kind (str) -- Search limiting filter, one of 'file', 'folder', 'connector'
- Returns
List of search results.
- Return type
list(SearchResult)
Files API#
- class edge.api.Folder#
Represents a folder in the Edge file system.
- Folder.list()#
List all file and folder names in this folder.
- Returns
List of File and Folder names.
- Return type
list(tr)
- Raises
NotFound --
- Folder.list_files()#
List all file names in this folder.
- Returns
List of file names.
- Return type
list(str)
- Raises
NotFound --
- Folder.list_folders()#
List all folder names in this folder.
- Returns
List of folder names.
- Return type
list(str)
- Raises
NotFound --
- Folder.open(name, open_with=None)#
Open an existing file or folder.
- Parameters
name (str) -- The name of the file or folder to open. May be a delimited series of folder names like "/this/folder/file" and we will walk all folders and open the final one.
open_with (str) -- Optional; name of an opener to use to open the object. Examples are "pandas" or "imageio", or "file" to get a file-like object giving access to the raw bytes. If not specified, an opener will be automatically picked for you.
- Return type
- Raises
NotFound --
ValueError --
- Folder.download(name, localpath=None)#
Download to a local file.
Currently, only files are supported.
- Parameters
name (str) -- Path/name of the remote object
localpath (str, optional) -- Download to this path locally. Defaults to the same basename in the current working directory.
- Folder.upload(destination, fp, overwrite=False)#
Upload a file to a folder.
- Parameters
destination (str) -- The path/name of the file upload destination.
fp (file-like) -- An object having callable read() returning bytes object.
overwrite (bool) -- If true, any existing file will be overwritten
- Raises
NotFound --
- Folder.make_folder(name)#
Make a new folder.
- Parameters
name (str) -- The name of the folder to create under this folder. May be a delimited series of folder names like "/this/folder/path" and we will ensure all folders in that series are created.
- Return type
- Folder.move(source, destination)#
Move a file or folder.
- Parameters
source (str) -- The file or folder in this folder to move, or path to source
destination (str) -- The path to the file to move to.
- Folder.rename(source, name)#
Move a file or folder.
- Parameters
source (str) -- The file or folder in this folder to rename.
name (str) -- The new name for the source object
- Folder.delete(object)#
Delete a file or folder.
- Parameters
object (str) -- The file or folder to delete, or path to object to delete
- class edge.api.File#
Represents a file in the Edge file system. You can get one of these via
Folder.open(filename)
. Please note files are read-only; to change a file's contents, upload (and overwrite) usingFolder.upload
.This class also has all file-like methods defined on Python's
io.RawIOBase
interface, appropriate to a read-only file.- property File.basename#
Name of the File object.
- property File.size#
Size of the File object
- File.read(size=-1)#
- File.write()#
- File.seek(offset, whence=0)#
Change stream position.
Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:
0 -- start of stream (the default); offset should be zero or positive
1 -- current stream position; offset may be negative
2 -- end of stream; offset is usually negative
Return the new absolute position.
- File.tell()#
Return current stream position.
- File.download_url()#
Data Sources API#
- class edge.api.EdgeData#
This object is available at
EdgeSession.sources
.- EdgeData.list()#
List names of stored external connections.
- Returns
Names of stored external connections.
- Return type
list(str)
- EdgeData.get(connection_id)#
Retrieve a stored connection object, given its name or UID.
- Parameters
connection_id (str) -- Name or UID of the connection to retrieve.
- Returns
The object corresponding to the retrieved connection.
- Return type
IConnection
- Raises
NotFound -- Raised if a connection with the given name is not found.
BadEntry -- If the connector definition or name is invalid.
- EdgeData.add(connection)#
Adds a connection to the database
- Parameters
connection (IConnection) -- A connector to add to the database.
- EdgeData.update(connection)#
Updates a connection in the database
- Parameters
connection (IConnection) -- A connector to update in the database.
- EdgeData.remove(connection_id)#
Delete a given connection from storage.
- Parameters
connection_id (str) -- Name or UID of the connection to remove.
SQL Data Connector#
- class edge.api.SQLConnection(name, title, host, port, db_kind, db_name, username, password)#
Represents a connection to a remote SQL data store.
You can access table contents via the .tables dictionary-like attribute.
Examples for use:
# List table names [1]: list(conn.tables) ['experiments', 'materials'] # Retrieve a table proxy object [2]: mytable = conn.tables['experiments'] # Fetch the table contents as a Pandas dataframe [3]: df = mytable.to_dataframe()
- property kind#
Connection kind, for discrimination
- property uid#
Connection unique ID
- property name#
Connection name, for internal use
- property title#
Title for display
- property engine#
SQL Engine
- property tables#
SQL Tables
- test(timeout=None)#
Test the connection to the external database
- Parameters
timeout (int, optional) -- The time in seconds to wait before reporting a connection as failed. If not specified, it defaults to a short timeout.
- Return type
ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS, or NO_CONNECTION)
- to_dict()#
Translates the connection as a dictionary, for serialization.
- Returns
Connection contents in dictionary form.
- Return type
Dict
S3 Data Connector#
- class edge.api.S3Connection(name, title, bucket, endpoint, access_key, secret_key, _verify=None)#
Represents a connection to a remote S3 bucket.
Use the conn.root attribute to access the contents as a file system.
Examples for use:
# List files and folders [1]: conn.root.list() ['myfile.jpg', 'mysubfolder'] # Open a file [2]: img = conn.root.open('myfile.jpg') # Open a "folder" in S3 [3]: myfolder = conn.root.open('myfolder') # List files in that subfolder [4]: myfolder.list() ['example.jpg'] # Download a file [5]: myfolder.download('example.jpg', localpath='example_downloaded.jpg')
- property kind#
Connection kind, for discrimination
- property uid#
Connection unique ID
- property name#
Connection name, for internal use
- property title#
Title for display
- property bucket#
Underlying S3 bucket
- property root#
Access an S3 bucket as a file system
- iter_prefixes(prefix=None)#
Search for key prefixes in the bucket.
Key prefixes can be used to organise the keys in a bucket in a structure that resembles a folder. The result contains strings between an optional prefix and and the first occurrence of the delimiter character, which is assumed to be "/". For example, if the bucket contains two keys named "folder1/myfile.txt" and "folder2/subfolder/myfile2.txt",
iter_prefixes()
finds the keys "folder1/" and "folder2/", whileiter_prefixes(prefix='folder2/')
finds "folder2/subfolder/".- Parameters
prefix (str, optional) -- Base prefix to use for searching other prefixes. Defaults to an empty prefix.
- Returns
An iterator yielding the prefixes found.
- Return type
iterator(str)
- list_prefixes(prefix=None)#
Search for key prefixes in the bucket.
Key prefixes can be used to organise the keys in a bucket in a structure that resembles a folder. The result contains strings between an optional prefix and and the first occurrence of the delimiter character, which is assumed to be "/". For example, if the bucket contains two keys named "folder1/myfile.txt" and "folder2/subfolder/myfile2.txt",
iter_prefixes()
finds the keys "folder1/" and "folder2/", whileiter_prefixes(prefix='folder2/')
finds "folder2/subfolder/".- Parameters
prefix (str, optional) -- Base prefix to use for searching other prefixes. Defaults to an empty prefix.
- Returns
A list containing the prefixes found.
- Return type
list(str)
- iter_keys(prefix=None)#
Search for keys in the bucket.
- Parameters
prefix (str, optional) -- Limit the search to keys starting with a given prefix.
- Returns
A list of the keys found in the bucket.
- Return type
iterator(str)
- list_keys(prefix=None)#
Search for keys in the bucket.
- Parameters
prefix (str, optional) -- Limit the search to keys starting with a given prefix.
- Returns
A list of the keys found in the bucket.
- Return type
list(str)
- test(timeout=None)#
Test the connection to the external filestore
- Parameters
timeout (int, optional) -- The time in seconds to wait before reporting a connection as failed. If not specified, it defaults to a short timeout.
- Return type
ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS, or NO_CONNECTION)
- open(key, open_with=None)#
Retrieve data from an S3 object as an appropriate Python object.
- Parameters
key (str) -- The S3 bucket key from which to retrieve data
open_with (str, optional) -- Opener to use, for example "pandas" or "imageio". Using "file" will give a file-like object with access to the raw bytes. If not specified, an appropriate opener will be selected for you.
- Returns
The most appropriate type found for the underlying data. If no specific type is found, a generic File is returned.
- Return type
Any
- Raises
ValueError -- Raised if requested type is not known.
NotFoundError -- Raised when the given key is not found in the bucket.
- to_dict()#
Translates the connection as a dictionary, for serialization.
- Returns
Connection contents in dictionary form.
- Return type
Dict
- class edge.api.S3Folder(_conn, _prefix)#
Represents a "folder" in an S3 bucket.
You can use this object to explore the contents of an S3 bucket interactively. S3 prefixes are mapped to folders, S3 keys are mapped to files.
[1]: s3conn.root.list_folders() ['folder1', 'folder2']
[2]: folder = s3conn.root.open('folder1') [3]: folder.list_files() ['myimg.jpg']
[4]: img = folder.open('myimg.jpg')
Private constructor
- list_folders()#
List all folder names in this folder.
- Returns
List of folder names.
- Return type
list(str)
- list_files(show_all=False)#
List all file names in this folder.
- Returns
list(str) -- List of file names.
show_all(bool) -- Also show files hidden by default. Currently this means file names which end with the string
$folder$
.
- list(show_all=False)#
List all file and folder names in this folder.
- Returns
list(tr) -- List of File and Folder names.
show_all(bool) -- Also show files hidden by default. Currently this means file names which end with the string
$folder$
.
- open(name, open_with=None)#
Retrieve an S3 object as an appropriate Python object.
- Parameters
name (str) -- The name of the file or folder to open.
open_with (str, optional) -- Opener to use, for example "pandas" or "imageio". Using "file" will give a file-like object with access to the raw bytes. If not specified, an appropriate opener will be selected for you.
- Returns
The most appropriate type found for the underlying data. If no specific type is found, a generic File is returned.
- Return type
Any
- Raises
ValueError -- Raised if requested type is not known.
NotFoundError -- Raised when the given key is not found in the bucket.
- download(name, localpath=None)#
Download to a local file.
Currently, only files are supported.
- Parameters
name (str) -- Name of the remote object
localpath (str, optional) -- Download to this path locally. Defaults to the same basename in the current working directory.
- class edge.api.S3File(bucket_object, context=None)#
File-like providing remote access to an S3 object for S3Connector.
Construct a new File object.
- Parameters
bucket_object (S3.Object) -- The object in a bucket this file represents
context (SSLContext) -- Optional SSL Context for S3 accesses - used in testing to disable SSL. Do not specify unless you have a strong need.
- property basename#
Name of the File object.
- property size#
Size of the File object
- readable()#
Return whether object was opened for reading.
If False, read() will raise OSError.
- seekable()#
Return whether object supports random access.
If False, seek(), tell() and truncate() will raise OSError. This method may need to do a test seek().
- writable()#
Return whether object was opened for writing.
If False, write() will raise OSError.
- tell()#
Return current stream position.
- seek(offset, whence=0)#
Change stream position.
Change the stream position to the given byte offset. The offset is interpreted relative to the position indicated by whence. Values for whence are:
0 -- start of stream (the default); offset should be zero or positive
1 -- current stream position; offset may be negative
2 -- end of stream; offset is usually negative
Return the new absolute position.
OpenAPI Data Connector#
- class edge.api.OpenAPIConnection(name, title, host, specs_url)#
A connector generating a REStful API from OpenAPI 3 specifications.
Example of use:
# Create a connector [1]: conn = OpenAPIConnector('petstore', 'Petstore Demo', ... host="http://petstore3.swagger.io", ... specs_url="https://petstore3.swagger.io/api/v3/openapi.json") # Generate a client allowing API access. # This may take a few seconds as it needs to generate source code. [2]: client = conn.generate_new_client() # Use the client to query the API [3]: pets = client.pet.findPetsByStatus()
Create a new OpenAPI3 connector.
- Parameters
name (Str) -- Connection name, must be unique
title (Str) -- Title as displayed in the Data App
host (Str) -- URL for the API host, including the http/https scheme
specs_url (Str) -- URL for the OpenAPI3 specification, including the http/https scheme.
- property kind#
Connection kind, for discrimination
- property uid#
Connection unique ID
- property name#
Connection name, for internal use
- property title#
Title for display
- property host#
Host for which we are generating an API
- property specs_url#
URL which points to the OpenAPI3 JSON spec file.
- to_dict()#
Translates the connection as a dictionary, for serialization.
- Returns
Connection contents in dictionary form.
- Return type
Dict
- test()#
Test the connection to the external data source
Note that the timeout should default to a short value, so a connection failure could also mean the network is slow or lagging.
- Parameters
timeout (int) -- The time in seconds to wait before reporting a connection as failed.
- Returns
ConnectionStatus (one of VALID_CONNECTION, BAD_CREDENTIALS,
or NO_CONNECTION)
- generate_new_client()#
Get a client allowing access to the remote API.
The remote API is available via methods on the returned object and its attributes. You can explore it by using e.g. IPython completion, or by using dir().
This method will generate Python files with the API binding, and cache them on disk. This is a fairly expensive process, so once a client is generated, you should try to use it for as long as possible. All files are cleaned up with the object becomes unreachable (or Python exits).