effectiveness

A study examined the effectiveness of app detection systems in identifying legitimate applications versus malware. It found that a simple phone backup app requesting access to storage, contacts, SMS, and call logs can be flagged by detection tools, and a more complex device-management tool raises even more alarms. Six Android detection systems, including Drebin, MalScan, and MaskDroid, were tested against benign apps from 49 Google Play categories, with over half flagged as suspicious. The least effective system, LAMD, incorrectly identified 80% of benign apps as malicious. Detection systems often match patterns from historical malware, which leads to legitimate apps being misclassified. The breadth of permissions requested by an app correlates with both malware and legitimate software, causing many benign applications to be wrongly flagged. The researchers noted that context is crucial for accurate identification, as similar API calls can be made by benign and malicious apps alike. The PRAXIS detection system operates in three stages: extracting permissions and components, organizing data into entries, sources, and effects, and assessing hypotheses to distinguish benign behaviors from harmful ones. When all stages are executed, PRAXIS misses about one in nine malware samples, but omitting the context stage results in a 95.3% undetected rate. The cost of summarizing app-specific functions is about [openai_gpt model="gpt-4o-mini" prompt="Summarize the content and extract only the fact described in the text bellow. The summary shall NOT include a title, introduction and conclusion. Text: A recent study has shed light on the challenges faced by app detection systems, particularly in distinguishing between legitimate applications and potential malware. The research highlights that a simple phone backup app, which requests access to storage, contacts, SMS, and call logs, can easily be flagged by various detection tools. In contrast, a more comprehensive device-management tool often raises even more alarms. This issue is compounded when these apps are analyzed by machine learning-based malware scanners, which frequently misidentify benign applications as harmful. Six prominent Android detection systems, including Drebin, MalScan, and MaskDroid, were tested against a curated set of benign apps sourced from 49 categories on Google Play. Alarmingly, more than half of these applications were flagged as suspicious. The least effective detector, an LLM-based system known as LAMD, incorrectly identified 80% of the apps as malicious. For developers and enterprises relying on these verdicts to gate app stores or deployment pipelines, this means facing a backlog primarily composed of legitimate software. Detectors match patterns from old malware The methodology employed by these detection systems often involves encoding features from app manifests and code into vectors, which are then analyzed using machine learning techniques. For instance, Drebin utilizes a support vector machine trained on these vectors, while others like MalScan and MaskDroid focus on analyzing call graphs and sensitive API subgraphs. These systems learn from historical malware patterns and score new applications based on their similarity to these past threats. Interestingly, the breadth of permissions requested by an app correlates not only with the likelihood of it being malware but also with legitimate software such as backup tools, antivirus programs, and parental controls. This overlap leads to a significant number of legitimate applications being misclassified as threats. Researchers succinctly summarized this phenomenon with the phrase, “deviation is not maliciousness.” The context in which an app operates plays a critical role in its classification. For example, a messaging application that reads incoming SMS messages behaves similarly to a cloud backup app that uploads files in the background. However, a banking app that covertly reads SMS messages to intercept one-time passwords (OTPs) poses a genuine threat, yet all three types of apps may trigger similar API calls. Make the code prove the guess The detection system developed by researchers, named PRAXIS, operates in three distinct stages. Initially, it extracts permissions and components from the app's manifest, sensitive API calls from bytecode, and function names from native libraries. This information is then analyzed by a large language model (LLM) to identify potential attack intents an attacker might exploit. In the second stage, PRAXIS organizes the data into entries, sources, and effects, filtering out those that do not form a reachable chain through various interactions. The final stage assesses the remaining hypotheses, distinguishing between benign behaviors triggered by user actions and those that operate silently in the background. One stage carries the detection When all three stages of PRAXIS are executed, the system misses approximately one in nine malware samples. However, if the context stage is omitted, the detection rate drops dramatically, with a staggering 95.3% of threats going undetected. This indicates that while the underlying behaviors and code evidence remain intact, the context is crucial for accurate identification. When presented with a straightforward judgment prompt, the model tends to overlook most potential threats, underscoring the importance of the final evaluation stage in the detection process. What is still open Despite the advancements in detection technology, several challenges remain. Summarizing app-specific functions incurs a cost of about [cyberseo_openai model="gpt-4o-mini" prompt="Rewrite a news story for a business publication, in a calm style with creativity and flair based on text below, making sure it reads like human-written text in a natural way. The article shall NOT include a title, introduction and conclusion. The article shall NOT start from a title. Response language English. Generate HTML-formatted content using tag for a sub-heading. You can use only , , , , and HTML tags if necessary. Text: A phone backup app asks for storage, contacts, SMS, and call logs. A device-management tool asks for more than that. Run either one past a machine learning malware scanner and it comes back flagged. Six Android detectors in wide research use, including Drebin, MalScan, and MaskDroid, produced that result on more than half the apps in a benign test set assembled from 49 Google Play categories. The worst performer, an LLM-based detector called LAMD, flagged 80% of them. Anyone gating an app store, an enterprise deployment, or a build pipeline on those verdicts is working a queue made mostly of legitimate software. Researchers at Singapore Management University and Nankai University built that set by pulling benign apps from AndroZoo, ranking them by declared dangerous permissions, and keeping the top 270. Every app in it declares at least nine. Their detector, PRAXIS, flagged 13%. Detectors match patterns from old malware Drebin encodes manifest and code features into a vector and trains a support vector machine on it. MalScan, MsDroid, and MaskDroid work over call graphs and sensitive-API subgraphs. Each one learns a distribution from past malware and scores new apps by fit. Permission breadth correlates with malware. It also correlates with backup tools, antivirus, parental controls, and anything that manages a device. The correlation carries the verdict, and legitimate software with a large permission surface lands on the wrong side of it. The researchers reduce the failure to four words: “deviation is not maliciousness.” The same operation flips depending on who triggered it and what the app advertises. A messaging app reads incoming SMS. A cloud backup app uploads files in the background with no user watching. Both are ordinary. A banking app that silently reads SMS and posts the contents to a remote server is running an OTP interception attack, and the API calls involved look the same in all three cases. Make the code prove the guess PRAXIS runs three stages. It pulls permissions and components from the manifest, sensitive API calls and implicit intent actions from bytecode through Soot, and function names from native libraries through Ghidra, then hands the pile to an LLM and asks for attack intents an attacker might pursue with them. PRAXIS overview (Source: Research paper) Each intent comes back as a guess with no pointer to code. The second stage casts it as an entry, a source, and an effect, retrieves candidate functions for each role, and keeps only the triples that reach one another in that order through call edges, inter-component transitions, or a shared field. A hypothesis with no reachable chain gets thrown out. The third stage judges what survived. A behavior reached only through a background broadcast runs without the user knowing. A behavior triggered from a screen gets checked against what that screen says it does. On a banking trojan impersonating Bank Mellat, that chain runs from an exported SMS_RECEIVED receiver through PDU parsing to an HttpURLConnection write, with a Flutter login form on top and no banking backend underneath. One stage carries the detection Run all three stages and PRAXIS misses about one malware sample in nine. Pull out the context stage and it misses 95.3%. The behaviors are still there. The code evidence is still there. Handed the same material with a plain judgment prompt, the model waves almost everything through. The final prompt carries most of the detection power. Everything upstream feeds it. What is still open Summarizing every app-specific function costs about $0.56 per app before the detection model spends anything. That price suits an analyst working a sample. It does not suit a store scanning submissions. The judgment stage reads UI text, app labels, package names, and decompiled identifiers. An attacker controls all four. A login screen that plausibly explains an SMS permission attacks the context reasoning directly, and no evaluation here covers an adversary who writes one. Then there is the training data. The learning baselines were retrained on apps from 2021 through 2023 and tested against samples running to 2026. PRAXIS runs on DeepSeek-V4-Pro, GPT-5.2, and GPT-5.4-mini, models whose training corpora plausibly include both those samples and the published Kaspersky and ESET reports the authors used to build their ground truth. The threats-to-validity section covers dataset breadth, annotator subjectivity, and decoding variance. Contamination goes unmentioned. Thirteen benign apps in every hundred still come back malicious. That is a review queue with a human in it, and a store gate built on the same number pulls working software off the shelf. Download: The High-Performance Team Playbook " temperature="0.3" top_p="1.0" best_of="1" presence_penalty="0.1" ].56 per app, which is manageable for analysts reviewing samples but impractical for app stores scanning submissions. Furthermore, the judgment stage relies on UI text, app labels, package names, and decompiled identifiers—all of which can be manipulated by attackers. A login screen that convincingly justifies SMS permission can directly challenge the context reasoning, revealing a gap in current evaluation methods. Additionally, the training data used for these detection models poses its own set of challenges. The learning baselines were retrained on apps from 2021 to 2023 and tested against samples projected to run until 2026. PRAXIS utilizes advanced models, including DeepSeek-V4-Pro and GPT-5.4-mini, which may have been trained on both the tested samples and existing reports from cybersecurity firms like Kaspersky and ESET. However, concerns regarding dataset breadth, annotator subjectivity, and potential contamination remain unaddressed. Ultimately, the findings indicate that even among benign apps, approximately 13% are still flagged as malicious. This creates a significant review workload for human analysts and raises questions about the reliability of app store gatekeeping processes that rely on similar detection rates." max_tokens="3500" temperature="0.3" top_p="1.0" best_of="1" presence_penalty="0.1" frequency_penalty="frequency_penalty"].56 per app, which is manageable for analysts but impractical for app stores. Additionally, the judgment stage relies on manipulable elements like UI text and app labels, creating vulnerabilities in detection methods. Concerns about the training data's breadth and potential contamination remain unaddressed, with approximately 13% of benign apps still flagged as malicious, complicating the review process for human analysts.