College Board pulls back from plan to score individual socioeconomic challenges

College Board has retreated on its plan to use algorithms to index students' socioeconomic challenges. The new plan from College Board revolves around providing contextual data on a student's neighborhood without scoring individuals.

The Environmental Context Dashboard (ECD), which was dubbed an "adversity score" in media accounts, aimed to highlight students that overcome challenges and achieved more with less. That effort, now called Landscape, has been scaled back as College Board retooled its efforts to avoid algorithms and explainability issues. Here's a look at Landscape's data methodology and guidelines for usage.

Also: Executive's guide to prescriptive analytics | Free PDF download: How to win with prescriptive analytics | Managing AI and ML in the enterprise 2019: Tech leaders expect more difficulty than previous IT projects Tech Pro Research

 In previous coverage I noted:

Now you can just imagine College Board's pickle here. The ECD score is transparent about what factors are included. But there's no explanation about how the score will be used. Every university may be different so explainability could fall to individual institutions. In addition, students won't know their ECD even though admissions officers will.

And should College Board be truly transparent about the ECD and disclose scores to students there are other questions to ponder. Would a student with a score of 90 be heartened or discouraged? Would a student with a score of 10 feel like a failure if he had a SAT of 1,000 yet was deemed privileged? What's the balance between psychology, algorithms and transparency?  

College Board's plan now revolves around providing context and information about high schools and neighborhoods without scoring individual students. For instance, the original dashboard wasn't going to be disclosed to students. Students and parents were questioning whether you could opt-out in the admissions process.

The College Board flap highlights how algorithm design and use of data will become more scrutinized over time. Given the high stakes of college admissions -- not to mention student debt -- College Board's use of data will be scrutinized.

In an FAQ about Landscape, College Board addressed the adversity score issue. It said:

No, this is not an adversity score. Landscape does not measure adversity and never will. It simply helps admissions officers better understand the high schools and neighborhoods applicants come from. It does not help them understand an applicant's individual circumstances—their personal stories, hardships, or home life. This is not the purpose of Landscape or the role of the College Board and it never will be.

And on the opt-in and disclosure to students issue, College Board noted:

Landscape does not contain student-level data and therefore doesn't require students to opt in. The only exception is the student-specific test scores. The test score in Landscape is based on the scores that students choose to send to colleges.

The moves could address complaints that swirled around the Environmental Context Dashboard. The biggest complaints revolved around transparency and how the score would be used as well as the inability to track personal and individual challenges.

In a press release with a headline of "College Board Announces Improved Admissions Resource," the organization said the changes were made from feedback from educators and families in recent months. College Board noted:

Colleges have long considered context about students' high schools and neighborhoods when making admissions decisions. But with more applications coming from more places, getting consistent information about every high school and neighborhood is challenging. Admissions officers using Landscape estimate they lack high school information for about 25% of all applications. Landscape presents consistent high school and neighborhood information so admissions officers can fairly consider each student.

The data in Landscape is similar to what was provided in the Environmental Context Dashboard. A few data points include:

• High school data on locale, population near school, senior class size, average SAT scores, advanced placement participation and performance and percent of students eligible for free and reduced-price lunch.
• Information pulled from multiple public sources compiling crime, income, and employment data with a reliance on the U.S. Census. 
• High school and neighborhood information relative to national and state benchmarks and averages. Information includes college attendance, household structure, median family income, housing stability, education level, and crime.
• College Board also provided appropriate usage guidelines for landscape use.

11 jobs teens are about to lose to robots

Related stories:

• How ethics can combat bias in AI
• Google says it will address AI, machine learning model bias with technology called TCAV
• Salesforce adds AI bias module to Trailhead
• IBM launches tools to detect AI fairness, bias and open sources some code
• Special report: Managing AI and ML in the enterprise (free PDF) TechRepublic
• Executive's guide to prescriptive analytics
• Research: Tech leaders are eager to implement prescriptive analytics