A website promising its AI service can accurately scan pictures of penises for signs of sexually transmitted infections is earning the ire of healthcare advocates and digital privacy experts, among many other critics. But while the internet (and Jimmy Fallon) have taken the makers of Calmara to task over the past week, it actually took two years to get here.
Where did the AI ‘intimacy bestie’ come from?
Back in 2022, the company HeHealth debuted itself as an online way to “get answers about your penis health in minutes.” To receive this information, the website uses a combination of questionnaires and what the company claims is a “65-96 percent accurate” AI screening tool allegedly trained on proprietary datasets to flag photographic evidence of various STIs, including genital warts, herpes eruptions, and syphilis. “Cancer” is also included in the list of scannable signs. If the results come back “positive”, HeHealth can then refer users to healthcare professionals for actual physical screenings, diagnoses, and treatment options. It’s largely flown under the radar since then, with only around 31,000 people reportedly using its allegedly anonymized, encrypted services over the last two years. And then came Calmara.
With a website overloaded with Gen Z-centric buzzwords, Calmara sells itself as women’s new “intimacy bestie,” offering to scan pictures of their potential sexual partners’ penises for indications of STIs. According to HeHealth CEO’s latest LinkedIn post, HeHealth and Calmara “are totally different products.” However, according to Calmara’s website, HeHealth’s owners are running Calmara, and it utilizes the same AI. Calmara also markets itself as (currently) free and “really in its element when focused on the D.”
In a March 19 reveal announcement, one “anonymous user” claimed Calmara is already “changing the conversation around sexual health.” Calmara certainly sparked a conversation over the last week—just not the one its makers likely intended.
A novelty app
Both Calmara’s and HeHealth’s fine print concede their STI judgments “should not be used as substitutes for professional medical advice, diagnosis, treatment, or management of any disease or condition.” There’s an obvious reason why this is not actually a real medical diagnosis tool, despite its advertising.
It doesn’t take an AI “so sharp you’d swear it aced its SATs” to remember that the majority of STIs are asymptomatic. In those cases, they definitely wouldn’t be visible in a photograph. What’s more, a preprint, typo-laden paper explaining Calmara’s AI indicates it was trained on an extremely limited image database that included “synthetic” photos of penises, i.e. computer-generated images. Meanwhile, determining its surprisingly accuracy is difficult to do—Calmara’s preprint paper says its AI is around 94.4-percent accurate, while the homepage says 95 percent. Scroll down a little further, and the FAQ section offers 65-to-90 percent reliability. Not a very encouraging approach to helping foster safe sex practices that would, presumably, require mutual, trustworthy statements about sexual health.
“On its face, the service is so misguided that it’s easy to dismiss it as satire,” sex and culture critic Ella Dawson wrote in a viral blog post last week. Calmara’s central conceit—that new intimate partners would be comfortable enough to snap genital photos for an AI service to “scan”—is hard to imagine actually playing out in real life. “… This is not how human beings interact with each other. This is not how to normalize conversations about sexual health. And this is not how to promote safer sex practices.”
No age verification
Given its specific targeting of younger demographics, Dawson told PopSci she believes “it’s easy to see how a minor could find Calmara in a moment of panic and use it to self-diagnose” which would constitute obvious legal issues, as well as ethical ones. For one, explicit images of minors could constitute sexual child abuse material, or CSAM. While Calmara expressly states its program shouldn’t be used by minors, it still lacks even the most basic of age verification protocols at the time of writing.
“Calmara’s lack of any age verification, or even a checkbox asking users to confirm that they are eighteen years of age or older, is not just lazy, it’s irresponsible,” Dawson concludes.
Dubious privacy practices
More to the point, simply slapping caveats across your “wellness” websites could amount to the “legal equivalent of magic pixie dust,” according to digital privacy expert Carey Lening’s rundown. While Calmara’s FAQ section is much vaguer on technical details, HeHealth’s FAQ page does state their services are HIPAA compliant because they utilize Amazon Web Services (AWS) “to collect, process, maintain, and store” data—which is technically true.
On its page dedicated to HIPAA regulations, AWS makes clear that there is no such thing as “HIPAA certification” for cloud service providers. Instead, AWS “aligns our HIPAA risk management program” to meet requirements “applicable to our operating model.” According to AWS, it utilizes “higher security standards that map to the HIPAA Security Rule” which enables “covered entities and their business associates” subject to HIPAA to use AWS for processing, maintaining, and storing protected health information. Basically, if you consent to use Calmara or HeHealth, you are consenting to AWS handling penis pictures—be them yours, or someone else’s.
[Related: A once-forgotten antibiotic could be a new weapon against drug-resistant infections.]
That said, Lening says Calmara’s makers may have failed to consider newer state laws, such as Washington’s My Health My Data Act, with its “extremely broad and expansive view of consumer health data” set to go into effect in late June. The first of its kind in the US, the My Health My Data Act is designed specifically to protect personal health data that may fall outside HIPAA qualifications.
“In short, they didn’t do their legal due diligence,” Lening contends.
“What’s frustrating from the perspective of privacy advocates and practitioners is not that they were ‘embracing health innovation‘ and ‘making a difference‘, but rather that they took a characteristic ‘Move Fast, Break Things’ kind of approach to the problem,” she continues. “The simple fact is, the [online] outrage is entirely predictable, because the Calmara folks did not, in my opinion, adequately assess the risk of harm their app can cause.”
Keep Calmara and carry on
When asked about these issues directly, Calmara and HeHealth’s founders appeared nonplussed.
“Most of the criticism is based on wrong information and misinformation,” HeHealth CEO and Calmara co-founder Yudara Kularathne wrote to PopSci last Friday, pointing to an earlier LinkedIn statement about its privacy policies. Kularathne added that “concerns about potential for anonymized data to be re-identified” are being considered.
On Monday, Kularathne published another public LinkedIn post, claiming to be at work addressing, “Health data and Personally Identifiable Information (PHI) related issues,” “CSAM related issues,” “communication related issues,” and “synthetic data related issues.”
“We are addressing most of the concerns raised, and many changes have been implemented immediately,” Kularathne wrote.
When reached for additional details, Calmara CEO Mei-Ling Lu avoided addressing criticisms in email, and instead offered PopSci an audio file from “one of our female users” recounting how the nameless user and her partner employed HeHealth’s (and now Calmara’s) AI to help determine they had herpes.
“[W]hile they were about to start, she realized something ‘not right’ on her partner’s penis, but he said: ‘you know how much I sweat, this is heat bubbles,’” writes Lu. After noticing similar “heat bubbles… a few days later,” Stacy and her partner consulted HeHealth’s AI scanner, which flagged the uploaded photos and directed them to healthcare professionals who confirmed they both had herpes.
To be clear, medical organizations such as the Mayo Clinic freely offer concise, accurate information on herpes symptoms, which can include pain or itching alongside bumps or blisters around the genitals, anus or mouth, painful urination, and discharge from the urethra or vagina. Symptoms generally occur 2-12 days after infection, and although many people infected with the virus display either mild or no symptoms, they can still spread the disease to others.
Meanwhile, Calmara’s glossy (NSFW) promotional, double entendre-laden video promises that it is “The PERFECT WEBSITE for HOOKING UP,” but no matter how many bananas are depicted, using AI to give penises a once-over doesn’t seem particularly reliable, enjoyable, or even natural.
The post Please think twice before letting AI scan your penis for STIs appeared first on Popular Science.
Articles may contain affiliate links which enable us to share in the revenue of any purchases made.
from Popular Science https://ift.tt/mlJNUtD
0 Comments