Motivation
The Ed-Fi ODS/API is a REST API that support interoperability of student data systems. The API definition, via the Ed-Fi Data Standard, is extensible: many large-scale or specialized implementations add their own local use cases that are not supported out of the box by the Ed-Fi Data Standard (Extensions). Furthermore, the Data Standard receives regular updates; sometimes these are merely additive, and from time to time there are breaking changes. These factors make it impossible to create a one-size fits all client library.
But, not all is lost: the ODS/API exposes its API definition using OpenAPI, and we can use Swagger Codegen to build a client library based on the target installation’s data model / API spec. The basic process of creating a C# code library (SDK) is described in Ed-Fi documentation at Using Code Generation to Create an SDK (Note: this link is for ODS/API 7.1, but the instructions are essentially the same for all versions).
But what about Python? Yes, Swagger Codegen supports Python output. But it is
not quite enough - you also need to manage authentication on your own. And,
running Swagger Codgen requires the Java Development Kit (JDK). The notes below
will walk through generating a client library with help from Docker (no local
install of the JDK required) and demonstrate basic usage of a simple
TokenManager class for handling authentication.
See
Ed-Fi-API-Client-Python
for a source code repository containing the TokenManager class listed below,
which might receive updates after this post has been published.
Generating an Ed-Fi Client Package
The Swagger Codegen tool is available as a pre-built Docker
image,
at repository swaggerapi/swagger-codegen-cli. We will use it to build a
client package for working with Ed-Fi Data Standard v5.0, which is available through
Ed-Fi ODS/API v7.1. The ODS/API Landing Page has links
to the Swagger UI-based “documentation” (UI on top of OpenAPI specification) for
all currently supported versions of the ODS/API. From there, we can find a link
to the specification
document.
The example shell commands use PowerShell, and they are easily adaptable to Bash
or another shell. The generated code will be in a new edfi-client directory.
Note that this repository’s .gitignore file excludes this directory from
source control, since the original intent of this repository is to provide
instructions, not a full-blown client. If you fork this repository and want to
create your own package, then you may wish to remove that line from .gitignore
so that you can keep your custom client code in your forked repository.
$url = "https://api.ed-fi.org/v7.1/api/metadata/data/v3/resources/swagger.json"
$outputDir = "./edfi-client"
New-Item -Path $outputDir -Type Directory -Force | out-null
$outputDir = (Resolve-Path $outputDir)
docker run --rm -v "$($outputDir):/local" swaggerapi/swagger-codegen-cli generate `
-i $url -l python -o /local
On my machine, this took about a minute to run. Here’s what we get as output:
> ls edfi-client
Directory: C:\source\Ed-Fi-API-Client-Python\edfi-client
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 11/27/2023 9:31 PM .swagger-codegen
d----- 11/27/2023 9:31 PM docs
d----- 11/27/2023 9:32 PM out
d----- 11/27/2023 9:31 PM swagger_client
d----- 11/27/2023 9:31 PM test
-a---- 11/27/2023 9:31 PM 786 .gitignore
-a---- 11/27/2023 9:31 PM 1030 .swagger-codegen-ignore
-a---- 11/27/2023 9:31 PM 359 .travis.yml
-a---- 11/27/2023 9:31 PM 1663 git_push.sh
-a---- 11/27/2023 9:31 PM 351139 README.md
-a---- 11/27/2023 9:31 PM 96 requirements.txt
-a---- 11/27/2023 9:31 PM 1811 setup.py
-a---- 11/27/2023 9:31 PM 69 test-requirements.txt
-a---- 11/27/2023 9:31 PM 149 tox.ini
We have code, we have tests, and even documentation. Here is a usage example from one of the auto-generated docs:
from __future__ import print_function
import time
import swagger_client
from swagger_client.rest import ApiException
from pprint import pprint
# Configure OAuth2 access token for authorization: oauth2_client_credentials
configuration = swagger_client.Configuration()
configuration.access_token = 'YOUR_ACCESS_TOKEN'
# create an instance of the API class
api_instance = swagger_client.AcademicWeeksApi(swagger_client.ApiClient(configuration))
id = 'id_example' # str | A resource identifier that uniquely identifies the resource.
if_match = 'if_match_example' # str | The ETag header value used to prevent the DELETE from removing a resource modified by another consumer. (optional)
try:
# Deletes an existing resource using the resource identifier.
api_instance.delete_academic_week_by_id(id, if_match=if_match)
except ApiException as e:
print("Exception when calling AcademicWeeksApi->delete_academic_week_by_id: %s\n" % e)
Converting to Poetry
I like to use Poetry for managing Python packages
instead of Pip, Conda, Tox, etc. Converting the requirements.txt file for use
in Poetry is quite easy with this PowerShell command (hat
tip):
Push-Location edfi-client
poetry init --name edfi-client -l Apache-2.0
@(cat requirements.txt) | %{&poetry add $_.replace(' ','')}
Pop-Location
(The default requirements.txt file has some unexpected spaces; the replace
command above strips those out).
Missing Token Generation
Note the line above with access_token = 'YOUR_ACCESS_TOKEN'. Swagger Codegen
requires you to bring your own token generation routine. We can build one using
portions of the client library itself. The ODS/API supports the OAuth 2.0 client
credentials flow, which generates an bearer-style access token. A basic HTTP
request for authentication looks like this:
POST /v7.1/api/oauth/token HTTP/1.1
Host: api.ed-fi.org
Content-Type: application/x-www-form-urlencoded
Accept: application/json
grant_type=client_credentials&client_id=YOUR CLIENT ID&client_secret=YOUR CLIENT SECRET
There are some variations in how these parameters can be passed, but this may be the most common / universal format, and this is what we will implement here.
Generated tokens are only good for so long; they expire. When a token expires,
it would be nice if we could recognize that and automatically call for a new
one, instead of encountering an error. The generated code does not support token
refresh, and does not have an obvious hook for how to do so. For a very clean
developer experience, the authentication and refresh mechanisms would be built
into the ApiClient class created by Swagger Codegen. But be warned: if you
rerun the generator, it will overwrite your customizations.
Someone with deeper Python expertise can probably come up with multiple ways to approach the refresh problem. This sample code handles token refresh very crudely, requiring the user of the code to detect the problem and try to re-authenticate. Perhaps a Context Manager implementation would help here.
Copy the following source code and paste it into a file called
token_manager.py inside the edfi-client/swagger_client directory.
import json
from datetime import datetime, timedelta
from swagger_client.rest import ApiException
from swagger_client.configuration import Configuration
from swagger_client.api_client import ApiClient
class TokenManager(object):
"""
Creates a new instance of the TokenManager.
Parameters
---------
token_url: str
The token URL for the Ed-Fi API.
configuration: Configuration
A list dictionary of configuration options for the RESTClientObject.
Must study the RESTClientObject constructor carefully to understand the
available options.
"""
def __init__(self, token_url: str, configuration: Configuration) -> None:
assert token_url is not None
assert token_url.strip() != ""
self.token_url: str = token_url
self.configuration: Configuration = configuration
self.client: ApiClient = ApiClient(self.configuration)
self.expires_at = datetime.now()
def _authenticate(self) -> None:
post_params = {
"grant_type": "client_credentials",
"client_id": self.configuration.username,
"client_secret": self.configuration.password
}
headers = {
"Content-Type": "application/x-www-form-urlencoded"
}
token_response = self.client.request("POST", self.token_url, headers=headers, post_params=post_params)
data = json.loads(token_response.data)
self.expires_at = datetime.now() + timedelta(seconds=data["expires_in"])
self.configuration.access_token = data["access_token"]
"""
Sends a token request and creates an ApiClient containing the returned access token.
Returns
------
ApiClient
an ApiClient instance that has already been authenticated.
"""
def create_authenticated_client(self) -> ApiClient:
self._authenticate()
return self.client
"""
Re-authenticates if the token has expired.
"""
def refresh(self) -> None:
if datetime.now() > self.expires_at:
self._authenticate()
else:
raise ApiException("Token is not expired; authentication failure may be a configuration problem.")
Demonstration
The following snippet demonstrates use the token manager with a simple token refresh mechanism. Note that this tries to delete an object that does not exist, therefore you should expect it to raise an exception with a 404 NOT FOUND message.
from swagger_client.configuration import Configuration
from swagger_client.token_manager import TokenManager
from swagger_client.api import AcademicWeeksApi
from swagger_client.rest import ApiException
BASE_URL = "https://api.ed-fi.org/v7.1/api"
config = Configuration()
config.username = "RvcohKz9zHI4"
config.password = "E1iEFusaNf81xzCxwHfbolkC"
config.host = f"{BASE_URL}/data/v3/"
config.debug = True
tm = TokenManager(f"{BASE_URL}/oauth/token", config)
api_client = tm.create_authenticated_client()
api_instance = AcademicWeeksApi(api_client)
try:
api_instance.delete_academic_week_by_id("bogus")
except ApiException as ae:
if ae.status == 401:
tm.refresh()
api_instance.delete_academic_week_by_id("bogus")
else:
raise
Posted with : Ed-Fi Alliance, General Programming