Emma Doe’s lawsuit alleged that Google has taken advantage of a dominant position it has held in schools since 2007 to push AI on kids and seize their data without meaningfully protecting them from the worst potential harms of its chatbot’s sycophantic design. In Chicago, CPS signed a contract with Google in 2020, “long before Gemini was rolled out,” Creswell found. Over the next two extensions of that contract, no updates were introduced to protect students’ privacy as AI advanced.
To Creswell, it remained “totally unclear” in contracts how Google’s processing of student data may have changed, beyond updates to the privacy policy for Google Workspace for Education.
Currently, that policy specifies that “we receive customer data through the Core Services and process it according to the school’s (the customer’s) instructions. This customer data includes anything submitted, stored, sent, or received through Core Services by you or your school.” Google collects that data on that activity, as well as account information “like name, email address and password” and location information, the policy said.
The policy didn’t discuss Gemini but noted that data from using core services helps maintain services, improve personalization, and increase security. And “the data we collect in the Additional Services is used across our services to deliver, maintain, and improve our services; develop new services; provide personalized services; measure performance; communicate with you; and protect Google, our users, and the public.”
To parents, the policy provides no reassurance that their kids’ data isn’t part of Google’s plan to dominate the AI industry.
“We should not have any child data being collected in a school setting being used for commercial purposes,” Creswell said.
Leave a Reply