Australia gets a national guide to help assess effectiveness of STEM initiatives

In a bid to standardise programs designed to increase female participation in science, technology, engineering, and mathematics (STEM), the Office of Women in STEM Ambassador has published a national guide to help those running these gender equity programs to evaluate the effectiveness of their initiatives.

The Evaluating STEM Gender Equity Programs guide is an online resource that provides advice, planning tools, and other guidance, and has been organised into five steps: Define, plan, design, execute, and share.

"We're trying to attract and keep women and girls in STEM … and that end goal is to have a diverse gender balance in the STEM workforce … and if we're trying to create that change, then we need to know what's working, and we need to know what's not working and how to improve that," guide author and research associate at the Office of the Women in STEM Ambassador Isabelle Kingsley told ZDNet.

"We're really hoping that anyone running a [STEM] program will be using this guide. One of the main things that is part of this guide is that evaluation is embedded from the beginning and not tacked on at the end."

The guide has been developed based on feedback from a pilot version that was released in May.

"What people said was that it was very comprehensive, but also too long. People said wouldn't it be great if there was a shorter and quick reference version ... that was one of the main things we did … taking that piece of work and condensing it down to a shorter version," Kingsley said.

Read also: Revisiting the conversation about tech diversity and inclusion in Australia

While the evaluation guide is not mandatory, Kingsley is optimistic it will foster consistency within STEM programs. The guide, for instance, was recently included as a condition as part of the recent round of the Australian government's Women in STEM and Entrepreneurship grants.

"The more people use a consistent way of evaluating we'll have a consistent and comparable suite of evaluation. The end goal really is to have some kind of repository where all of these evaluations live, and it's consistent and comparable across different programs," Kingsley said.

Last month, a study undertaken by a researcher at the Australian National University (ANU) revealed there was still little evidence of how effective these STEM initiatives are.

The research, published in the Australian Journal of Social Issues, identified there were 337 gender equity programs and initiatives offered nationally, but only seven provided publicly facing evaluation data that went beyond self-reports of satisfaction and enjoyment.

"The myriad initiatives show a concerted effort to engage girls and women in STEM, but the absence of any meaningful evidence of impact means we simply do not know whether these initiatives are benefiting girls and women and achieving the desired policy outcomes or not," wrote Merryn McKinnon, research author and senior lecturer at the ANU Centre for the Public Awareness of Science.

Some of these initiatives that were examined as part of the study included the establishment of the federal government's Women in Science, Engineering, and Technology (WISET) advisory group in 1995, Women in STEM Entrepreneurship (WISE) grant program in 2015, and the Decadal Plan for Women in STEM during the 2018-2019 Budget.

While facing Senate Estimates, Australia's Chief Scientist Dr Alan Finkel recently admitted there was still a long way to go before the country would reach parity.

"It's influenced enormously by home and parents. The decision for a young woman is influenced enormously by peer pressure, teachers, and role models, and there is a lot of effort … to try to redress it, but it takes time," he said.

"There has been progress, but the rate of progress is very, very small."

He said the STEM workforce graduate figure for females was around 29% in 2016 and 27% five years earlier.

"It's a move in the right direction, but a very small move," he said.

During Estimates, Finkel was also questioned whether parity would be reached by 2091.

"Correct -- hopefully earlier," he said. "It's not exactly clear what the real goal is. Is it parity in every single discipline, which is unreasonable, or is it gender parity across the whole totality of all the STEM disciplines? Or is it gender parity across all the professional disciplines, whether it is STEM or non-STEM?"

Elsewhere, the University of Technology, Sydney (UTS) has been undertaking work for the last two years to develop a baseline assessment to help companies independently validate ethical AI.

As part of the work, the research team has reviewed more than 175 frameworks and guidelines from around the world and refined these into four categories of ethical principles: Transparency, accountability, fairness, and security and privacy.

These principles were recently used to help workforce platform firm Reejig assess its AI algorithm and processes.

Read more: AI and ethics: The debate that needs to be had

"We all know AI ethics is super important and there are plenty of documents talking about various principles … and when Reejig approached us, the question from them was, 'Yes, there are so many guidelines, but which one should we follow? And how do we know we're following them?'" UTS Data Science executive director Fang Chen told ZDNet.

"It's like knowing you have an exam but how do you pass those exams to earn a credit, so we took it as a research challenge to identify those core principles and create this framework to do this quantitative and qualitative analysis.

"Against each of the principles, there are a whole set of questions and demonstrations you need to provide so you meet the criteria. We worked with Reejig through each one … and made sure they satisfied each criteria."

Fang believes the assessment could be used as a guide for potentially broader work being carried out around ethical AI.

"We should bring everyone on board together. Even though there is no particular one organisation or one association, which can deal with this broad issue, at UTS we're very happy to open the door and encourage everyone to come on board, and speak to us about what we could to formulate a more general assessment to tackle this issue," she said.

RELATED COVERAGE

Australia's Women in STEM Ambassador says the country's efforts to diversify is glacial

But she thinks relentless pressure will eventually bring about a culture change.

Western Australia launches its first STEM skills strategy

To create a globally competitive and STEM-fluent workforce in Western Australia.

Three women in tech keeping the gender conversation going

It's not enough to just talk about gender equality and move on -- it needs to be an ongoing movement.

Seeking diversity in Australia's intelligence and cybersecurity workforce

As analysts call for a review of Australia's intelligence agency staffing, aimed at increasing diversity, CyberCX sets up a cyber scholarship for women.