10 posts tagged with "data-and-analytics"

Data engineering gurus Joe Reis and Matthew Housley once again led a closing town hall at Data Day Texas. Rather than opining from the front, they turned the session over to the wisdom of the crowd. Housley seeded the conversation with a single question – “what is the elephant in the room?” – and the room was ready with an answer: AI. In particular: what is AI going to do to my job?

Given a room full of strangers, some participants were remarkably open about their fears. Perhaps knowing that the audience is composed of fellow data geeks helped to establish a sense of vulnerability. These fears were being expressed by the people who, in theory, should be the ones developing expertise in using AI tooling. But that’s how disruptive the technologies may be: even the data experts are uncertain and afraid.

Notes from three talks on data cataloging at the recent Data Day Texas 2025

• The Meta Grid
• Investing in Semantics and Knowledge
• We Are All Librarians

A strange thing about my job is that, although we're all about supporting K-12 education data interoperability, we don't actually work with any K-12 data. We build software, and others use it to collect data from disparate data sources into a single, unified, and standardized data set. But that does't stop me from thinking about how data should be used.

On a flight out to the #STATSDC2023 conference hosted by the National Center for Educational Statistics (my first time at this event), I finally wrote down my personal principles for ethical / responsible use of data and AI. Many have written about responsible use of data; there is nothing ground breaking here. Yet it feels meaningful, even if only for myself, to acknowledge "out loud" the values and principles that I wish to hold myself accountable for whenever I do use data, encourage others to make use of data, allow my own data to be used, etc.

Are algorithms doomed to be racist and harmful, or is there a legitimate role for them in a just and equitable society?

Algorithms have been causing disproportionate harm to low- and middle-income individuals, especially people of color, since long before this current age of machine learning and artificial intelligence. Two cases in point: neighborhood redlining and credit scores. While residential redlining was a deliberately racist anti-black practice [1], FICO-based credit scoring does not appear to have been created from a racist motive. By amplifying and codifying existing inequities, however, the credit score can easily become another tool for racial oppression [2].

Still, with appropriate measures in place, and a bit of pragmatic optimism, perhaps we can find ways to achieve the scalability/impartiality goals of algorithms while upholding true equity and justice.

Advances in the availability and breadth of data over the past few decades have enabled the rapid and unregulated deployment of statistical algorithms that aim to predict and thereby influence the course of human behavior. Most are designed to promote the corporate bottom line, not the welfare of the people. Those that aim to promote the common good run the danger of straying into authoritarian suppression of freedoms. Regardless of intention, these algorithms often reinforce existing social inequities or present a double-edged sword, with potential for positive use weighed against potential for misuse.

Soon the Ed-Fi Alliance will release version 2.0 of our Analytics Middle Tier, welcoming it to the "big kids' table" as a fully supported add-on to the ODS database.

When we started this project in 2018, it was with the aim of creating a production-ready proof-of-concept (a seeming oxymoron, we know) for simplifying the Ed-Fi Data Model, thus making the ODS more useful for reporting and ad hoc analytics exploration (see From Diagrams & Definitions: Solving the Analytics Reporting Gap). As such, it was released on the Ed-Fi Exchange instead of being bundled into the core ODS/API platform.

Continue reading on ed-fi.org…

The most popular request at the 2018 Ed-Fi Summit’s tech town hall was for an option to run the Ed-Fi ODS / API on an open source database solution. Historically, the Operational Data Store (ODS) database has been developed on Microsoft SQL Server, matching the preference of educational agencies that rely on heavily discounted licensing terms for on-premises operation of SQL Server. The advent of cloud-based hosting has changed that dynamic, especially since Microsoft ended the "bring your own license" (BYOL) practice.

One implication of that change is that educational agencies wishing to use SQL Server may need to pay full price when using non-Azure managed services for SQL Server; however, even with BYOL, managed services with SQL Server does cost more than other database platforms. So, with the help of an Ed-Fi Special Interest Group, we narrowed the field to one alternative database platform (for now…).

continue reading on ed-fi.org...

If data lands in the ODS and no one uses it, does it empower educators?

Ed-Fi Community members are increasingly leveraging the Ed-Fi ODS as a source of data for business intelligence (BI) solutions, while many continue to develop it as a solution for compliance reporting. While the Ed-Fi vendor community provides many options for analytics based on the ODS data, some end-users wish to create their own custom reports and perform their own ad hoc analysis. The Analytics Middle Tier and the two Analytics Starter Kits recently published to the Ed-Fi Exchange aim to help this group by simplifying the ODS data model, provisioning support for role-based data access, and providing sample visualizations. These solutions aim to empower those IT staff who are empowering their educators and administrators.

continue reading on ed-fi.org...

According to Amazon Web Services (AWS) CEO Andy Jassy, at his keynote Wednesday morning, I am one of around 53,000 people from all over the world who have come out to the annual AWS re:Invent conference in Las Vegas. We come together from myriad industries and with interests that span the full range from product development to operations to account management. My personal learning objectives for the week are to deepen my understanding, and think about implications for the Ed-Fi tech stack, of four concepts:

• Data lakes
• Serverless
• Business intelligence
• .NET on AWS

continue reading on ed-fi.org...

Have you ever tried to write a query using the Ed-Fi ODS for reporting or analytics? To say that it is challenging is to use the mildest language. The Data Standard documentation in Tech Docs is top notch. Nevertheless, going from diagrams and definitions to actual query code for, let’s say, each student’s average math grade during a grading period, is not a trivial exercise.

continue reading on ed-fi.org...