In Unpopular Opinion: It’s harder than ever to be a good software engineer, author Juraj Malencia opines that “good engineering” equates to “bring[ing] maximum value in achieving a goal.” This is a comforting thought for those of us periodically wondering if we’re still good software or data engineers, though we might not be up-to-date on the latest hype.

Pointing out that “engineering does not equal programming”, implying that we can bring great “engineering” (solutioning) value without writing a line of code, he presents a modified version of a venn diagram on how people in various roles spend their time between programming, alignment, people, and “other”. I was bemused to note I currently sit squarely in the category labeled “beware” 🤨. A position that may continue throughout this year, but will need to change over time.

Motivation

The Ed-Fi ODS/API is a REST API that support interoperability of student data systems. The API definition, via the Ed-Fi Data Standard, is extensible: many large-scale or specialized implementations add their own local use cases that are not supported out of the box by the Ed-Fi Data Standard (Extensions). Furthermore, the Data Standard receives regular updates; sometimes these are merely additive, and from time to time there are breaking changes. These factors make it impossible to create a one-size fits all client library.

But, not all is lost: the ODS/API exposes its API definition using OpenAPI, and we can use Swagger Codegen to build a client library based on the target installation’s data model / API spec. The basic process of creating a C# code library (SDK) is described in Ed-Fi documentation at Using Code Generation to Create an SDK (Note: this link is for ODS/API 7.1, but the instructions are essentially the same for all versions).

A strange thing about my job is that, although we’re all about supporting K-12 education data interoperability, we don’t actually work with any K-12 data. We build software, and others use it to collect data from disparate data sources into a single, unified, and standardized data set. But that does’t stop me from thinking about how data should be used.

On a flight out to the #STATSDC2023 conference hosted by the National Center for Educational Statistics (my first time at this event), I finally wrote down my personal principles for ethical / responsible use of data and AI. Many have written about responsible use of data; there is nothing ground breaking here. Yet it feels meaningful, even if only for myself, to acknowledge “out loud” the values and principles that I wish to hold myself accountable for whenever I do use data, encourage others to make use of data, allow my own data to be used, etc.

Full write-up of experiments in MongoDB connection string tuning with Ed-Fi Project Meadowlark: RND-380: MongoDB Connection Pooling

Design document describing approaches and appropriateness of in-memory and distributed caching for improving the performance of referential integrity checks in the Ed-Fi Meadowlark API: Resource Caching for Performance Improvements

Design document describing an event-driven approach for offline cascading updates in the Ed-Fi Meadowlark project’s document storage system: Design for Offline Cascading Updates

On July 13, a new statue was placed in the U.S. Capitol: Mary McLeod Bethune. Reading the news, I knew that I had heard this name - yet knew nothing about her. Who was this woman, the first African American to be so honored in the Hall of Statues?

Born into a large family on her parents’ farm in 1875 (she was the fifteenth child), she was taught early to look to the Bible for guidance and comfort, despite the family’s illiteracy. With help from a benefactress, she enrolled in school at the age of ten and eventually went on to collegiate study. Oft quoted as saying, “[t]he whole world opened up to me when I learned to read,” she went on to live an exceptional life of courage and action on behalf all people, most particularly her fellow African Americans and especially women of color.

As education agencies increasingly look for cost-savings and scalability benefits through adoption of hybrid or cloud-only IT infrastructures, it is natural to ask: what should an ideal Ed-Fi installation look like in the cloud?

The Ed-Fi Tech Congress in Phoenix, of April 2018, was a sink or swim moment for me, as I had just started working for the Ed-Fi Alliance. Among the first people I met was a representative from one of the big technology companies. The conversation quickly turned to the question of how to deal with data when the vendor would not send it directly into the Ed-Fi ODS/API. He asked me, “why not just put it in a data lake?” To which I had no reply. Nearly four years later, at last I can give a reasonable reply.

Prompted by a class I’m taking, I decided to try running Python from Windows Subsystem for Linux (WSL; actually, WSL2 to be specific). Installing Python in Ubuntu on Windows was relatively easy, though I did run into a couple of little problems with running poetry. Bigger challenge: running graphical user interfaces (GUIs) from WSL. Here are some quick notes from my experience.

Screenshot shows that I’m running Windows 10, and shows a small GUI window opened from both Powershell and from Bash using the same Python script.