tag:blogger.com,1999:blog-11769492575416861272024-03-08T13:42:06.951-05:00Google Online Security BlogThe latest news and insights from Google on security and safety on the Internet.Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.comBlogger573125tag:blogger.com,1999:blog-1176949257541686127.post-2791548541032543682024-03-04T14:00:00.022-05:002024-03-08T05:12:51.193-05:00Secure by Design: Google’s Perspective on Memory Safety<span class="byline-author">Alex Rebert, Software Engineer, Christoph Kern, Principal Engineer, Security Foundations</span><div><br /></div><div><span class="byline-author"><br /></span></div><div><span class="byline-author"><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Google’s Project Zero </span><a href="https://googleprojectzero.blogspot.com/2022/04/the-more-you-know-more-you-know-you.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">reports</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that </span><a href="https://www.memorysafety.org/docs/memory-safety/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">memory safety vulnerabilities</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">—security defects caused by subtle coding errors related to how a program accesses memory—have been "the standard for attacking software for the last few decades and it’s still how attackers are having success". Their analysis shows two thirds of 0-day exploits detected in the wild used memory corruption vulnerabilities. Despite substantial investments to improve memory-unsafe languages, those vulnerabilities continue to top the </span><a href="https://cwe.mitre.org/top25/archive/2023/2023_kev_list.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">most commonly exploited vulnerability classes</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In this post, we share our perspective on memory safety in a </span><a href="https://research.google/pubs/pub53121/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">comprehensive whitepaper</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. This paper delves into the data, challenges of tackling memory unsafety, and discusses possible approaches for achieving memory safety and their tradeoffs. We'll also highlight our commitments towards implementing several of the solutions outlined in the whitepaper, most recently with a </span><a href="https://security.googleblog.com/2024/02/improving-interoperability-between-rust-and-c.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">$1,000,000 grant to the Rust Foundation</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, thereby advancing the development of a robust memory-safe ecosystem.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"></p>
<h1 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 24pt;"><span style="color: #4285f4; font-family: Roboto, sans-serif; font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Why we’re publishing this now</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">2022 marked the 50th anniversary of memory safety vulnerabilities. Since then, memory safety risks have grown more obvious. Like others', Google's internal vulnerability data and research show that memory safety bugs are widespread and one of the leading causes of vulnerabilities in memory-unsafe codebases. Those vulnerabilities endanger end users, our industry, and the broader society. We're encouraged to see governments also taking this issue seriously, as with the U.S. Office of the National Cyber Director publication of a </span><a href="https://www.whitehouse.gov/wp-content/uploads/2024/02/Final-ONCD-Technical-Report.pdf" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">paper</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> on the topic last week.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">By sharing our insights and experiences, we hope to inspire the broader community and industry to adopt memory-safe practices and technologies, ultimately making technology safer.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"></p>
<h1 dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span style="color: #4285f4; font-family: Roboto, sans-serif; font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Our perspective</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">At Google, we have decades of experience addressing, at scale, large classes of vulnerabilities that were once similarly prevalent as memory safety issues. Our approach, which we call “</span><a href="https://research.google/pubs/pub53116/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Safe Coding</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">”, treats vulnerability-prone coding constructs themselves as hazards (i.e., independently of, and in addition to, the vulnerability they might cause), and is centered around ensuring developers do not encounter such hazards during regular coding practice.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="color: #434343;">Based on this experience, we expect that high assurance memory safety can only be achieved via a </span></span><span style="background-color: white; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><a href="https://blog.google/technology/safety-security/tackling-cybersecurity-vulnerabilities-through-secure-by-design/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Secure-by-Design approach</span></a></span><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> centered around comprehensive adoption of languages with rigorous memory safety guarantees. As a consequence, we are considering a gradual transition towards memory-safe languages like Java, Go, and Rust.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Over the past decades, in addition to large Java and Go memory-safe codebases, Google has developed and accumulated hundreds of millions of lines of C++ code that is in active use and under active, ongoing development. This very large existing codebase results in significant challenges for a transition to memory safety:</span></p>
<ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">We see no realistic path for an evolution of C++ into a language with rigorous memory safety guarantees that include temporal safety.</span></p></li>
<li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">A large-scale rewrite of all existing C++ code into a different, memory-safe language appears very difficult and will likely remain impractical.</span></p></li></ul>
<div><span style="color: #434343; font-family: Roboto, sans-serif;"></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 1pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We consider it important to complement a transition to memory safe languages for new code and particularly at-risk components with safety improvements for existing C++ code, to the extent practicable. We believe that substantial improvements can be achieved through an incremental transition to a partially-memory-safe C++ language subset, augmented with hardware security features when available. For instance, see </span><a href="https://bughunters.google.com/blog/6368559657254912/llvm-s-rfc-c-buffer-hardening-at-google" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">our work improving spatial safety in GCP's networking stack</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p>
<h1 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 24pt;"><span style="color: #4285f4; font-family: Roboto, sans-serif; font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Our investments in memory-safe languages</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We are actively investing in many of the solutions outlined in our whitepaper and in our </span><a href="https://www.regulations.gov/comment/ONCD-2023-0002-0074" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">response</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to the </span><a href="https://www.regulations.gov/document/ONCD-2023-0002-0001" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">US Federal Government’s RFI on Open Source Software Security</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p>
<ul style="margin-bottom: 1pt; margin-top: 1pt; padding-inline-start: 48px;">
<li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">Android</span><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> has written several components in Rust over the last few years, leading to </span><a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">compelling security improvements</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">. In Android’s Ultra-wideband (UWB) module, this has improved the security of the module while also reducing the memory usage and inter-procedural calls. </span></p></li></ul><ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;">
<li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">Chrome</span><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> has </span><a href="https://groups.google.com/a/chromium.org/g/chromium-dev/c/UhwVDk4HZFA/m/UAA2D96QBAAJ" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">started shipping some features in Rust</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">; in one case, Chrome was able to move its QR code generator out of a sandbox by adopting a new memory-safe library written in Rust, leading to </span><span style="font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">both</span><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> better security and better performance.</span></p></li></ul>
<ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Google recently announced a </span><a href="https://security.googleblog.com/2024/02/improving-interoperability-between-rust-and-c.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">$1,000,000 grant to the Rust foundation</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> to enhance interoperability with C++ code. This will facilitate incremental adoption of Rust in existing memory-unsafe code bases, which will be key to enabling even more new development to occur in a memory-safe language. Relatedly, we are also working on addressing </span><a href="https://bughunters.google.com/blog/4805571163848704/llvm-cfi-and-cross-language-llvm-cfi-support-for-rust" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">cross-language attacks</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> that can occur when mixing Rust and C++ in the same binary.</span></p></li></ul>
<ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre-wrap;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Google is investing in building the memory-safe open-source ecosystem through </span><a href="https://www.memorysafety.org/" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">ISRG Prossimo</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> and </span><a href="https://alpha-omega.dev/" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">OpenSSF’s Alpha-Omega project</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">. Back in 2021, we funded </span><a href="https://www.memorysafety.org/blog/supporting-miguel-ojeda-rust-in-linux/" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">efforts to bring Rust to the Linux Kernel</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">, which is now enabling us to write </span><a href="https://lore.kernel.org/lkml/20231101-rust-binder-v1-0-08ba9197f637@google.com/" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">memory-safe drivers</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">. This funding is also going towards providing alternatives or upgrades to key open-source libraries in a memory-safe language, such as providing a </span><a href="https://www.memorysafety.org/blog/rustls-and-rust-for-linux-funding-openssf/" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">memory safe TLS implementation</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">.</span></p></li></ul><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"></p>
<p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We know that memory safe languages will not address every security bug, but just as our efforts to </span><a href="https://bughunters.google.com/blog/5896512897417216/a-recipe-for-scaling-security" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">eliminate XSS attacks through tooling showed</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, removing large classes of exploits both directly benefits consumers of software and allows us to move our focus to addressing further classes of security vulnerabilities.</span></p>
<p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">To access the full whitepaper and learn more about Google's perspective on memory safety, visit </span><a href="https://research.google/pubs/secure-by-design-googles-perspective-on-memory-safety/" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://research.google/pubs/secure-by-design-googles-perspective-on-memory-safety/</span></a><span style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> </span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-22878358204762553752024-02-06T20:44:00.004-05:002024-02-29T19:02:20.778-05:00Piloting new ways of protecting Android users from financial fraud <span class="byline-author">Posted by Eugene Liderman, Director of Mobile Security Strategy, Google</span>
<p>
<a href="https://services.google.com/fh/files/misc/android-enterprise-security-paper-2023.pdf">From its founding</a>, Android has been guided by principles of openness, transparency, safety, and choice. Android gives you the freedom to choose which device best fits your needs, while also providing the flexibility to download apps from a variety of sources, including preloaded app stores such as the Google Play Store or the Galaxy Store; third-party app stores; and direct downloads from the Internet.<br /><br />Keeping users safe in an open ecosystem takes sophisticated defenses. That’s why Android provides <a href="https://blog.google/products/android/how-android-helps-you-stay-safe-from-mobile-fraud-apps/">multiple layers of protections</a>, powered by AI and backed by a large dedicated security & privacy team, to help to protect our users from security threats while continually making the platform more resilient. We also provide our users with numerous built-in protections like Google Play Protect, the world’s most widely deployed threat detection service, which actively scans over 125 billion apps on devices every day to monitor for harmful behavior. That said, our data shows that a disproportionate amount of bad actors take advantage of select APIs and distribution channels in this open ecosystem.
</p>
<p>
<b>Elevating app security in an open ecosystem</b>
</p>
<p>
While users have the flexibility to download apps from many sources, the safety of an app can vary depending on the download source. Google Play, for example, carries out rigorous operational reviews to ensure app safety, including proper high-risk API use and permissions handling. Other app stores may also follow <a href="https://www.gov.uk/government/publications/code-of-practice-for-app-store-operators-and-app-developers">established policies and procedures</a> that help reduce risks to users and their data. These protections often include requirements for developers to declare which permissions their apps use and how developers plan to use app data. Conversely, standalone app distribution sources like web browsers, messaging apps or file managers – which we commonly refer to as Internet-sideloading – do not offer the same rigorous requirements and operational reviews. Our data demonstrates that users who download from these sources today face unusually high security risks due to these missing protections.<del><br /></del>
</p>
<p>
We recently launched <a href="https://security.googleblog.com/2023/10/enhanced-google-play-protect-real-time.html">enhanced Google Play Protect real-time scanning</a> to help better protect users against novel malicious Internet-sideloaded apps. This enhancement is designed to address malicious apps that leverage various methods, such as AI, to avoid detection. This feature, now deployed on Android devices with Google Play Services in India, Thailand, Singapore and Brazil, has already made a significant impact on user safety.
</p>
<p>
As a result of the real-time scanning enhancement, Play Protect has identified 515,000 new malicious apps and issued more than 3.1 million warnings or blocks of those apps. Play Protect is constantly improving its detection capabilities with each identified app, allowing us to strengthen our protections for the entire Android ecosystem.
</p>
<p>
<b>A new pilot to combat financial fraud</b>
</p>
<p>
<br />Cybercriminals continue to invest in advanced financial fraud scams, costing consumers more than $1 trillion in losses. According to the <a href="https://www.newswire.com/news/global-losses-to-scammers-exceed-1-trillion-as-1-in-4-lose-money-to-22145725">2023 Global State of Scams Report</a> by the Global Anti-Scam Alliance, 78 percent of mobile users surveyed experienced at least one scam in the last year. Of those surveyed, 45 percent said they’re experiencing more scams in the last 12 months. The Global Scam Report also found that scams were most often initiated by sending scam links via various messaging platforms to get users to install malicious apps and very often paired with a phone call posing to be from a valid entity.<br />
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQ88Ifjnbt6zm9FgO1q6FkuffMb2Hd4lmKnMC7PvGa3PvH-9n7Gohaa1lS2EDE1kKQkQlUXQXiXBQBEBSuL7MrjtvpKZRijKAMlrh0hZ3pyzju_q9hQozsE2wduFEgR8I8xivI1KJeq1O1ntlv98BSlmhO3ueVX_hUYnbwCf8cDnKRbZDrycZUkW7QZQG0/s1600/high-risk.png" style="display: block; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="711" data-original-width="1919" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQ88Ifjnbt6zm9FgO1q6FkuffMb2Hd4lmKnMC7PvGa3PvH-9n7Gohaa1lS2EDE1kKQkQlUXQXiXBQBEBSuL7MrjtvpKZRijKAMlrh0hZ3pyzju_q9hQozsE2wduFEgR8I8xivI1KJeq1O1ntlv98BSlmhO3ueVX_hUYnbwCf8cDnKRbZDrycZUkW7QZQG0/s1600/high-risk.png" /></a></div>
<p>
Scammers frequently employ <a href="https://www.businessthink.unsw.edu.au/articles/cracking-the-code-why-people-fall-scams">social engineering tactics</a> to deceive mobile users. Using urgent pretenses that often involve a risk to a user’s finances or an opportunity for quick wealth, cybercriminals convince users to disable security safeguards and ignore proactive warnings for potential malware, scams, and phishing. We’ve seen a large percentage of users ignore, or are tricked into dismissing, these proactive Android platform warnings and proceed with installing malicious apps. This can lead to users ultimately disclosing their security codes, passwords, financial information and/or transferring funds unknowingly to a fraudster.
</p>
<p>
To help better protect Android users from these financial fraud attacks, we are piloting enhanced fraud protection with Google Play Protect. As part of a continued strategic partnership with the Cyber Security Agency of Singapore (CSA), we will launch this first pilot in Singapore in the coming weeks to help keep Android users safe from mobile financial fraud.
</p>
<p>
This enhanced fraud protection will analyze and automatically block the installation of apps that may use sensitive permissions frequently abused for financial fraud when the user attempts to install the app from an Internet-sideloading source (web browsers, messaging apps or file managers). This enhancement will inspect the permissions the app declared in real-time and specifically look for four permission requests: RECEIVE_SMS, READ_SMS, BIND_Notifications, and Accessibility. These permissions are frequently abused by fraudsters to intercept one-time passwords via SMS or notifications, as well as spy on screen content. Based on our analysis of major fraud malware families that exploit these sensitive permissions, we found that over 95 percent of installations came from Internet-sideloading sources.
</p>
<p>
During the upcoming pilot, when a user in Singapore attempts to install an application from an Internet-sideloading source and any of these four permissions are declared, Play Protect will automatically block the installation with an explanation to the user.
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGwZqITQ_mnZDHhjXjcobBqOhIknBtkz-XKhy9u2kcxFkTDO0f4rAU0CZLQTzO09CcucNG5BMoP1lODBjpb7D6QJ7hhZjSzHLTmRYUm8IdGF2FfF5dq2lg73l0zaHclqKYzYN_ddLNq1eQ3pkVXNqzNJCA0c25nzyYSfZQcaEbxIdnl4fledx_cZzoIdxM/s1087/HRA%20gif%20%281%29.gif" style="display: block; padding: 1em 0px; text-align: center;"><img alt="" border="0" data-original-height="1087" data-original-width="512" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGwZqITQ_mnZDHhjXjcobBqOhIknBtkz-XKhy9u2kcxFkTDO0f4rAU0CZLQTzO09CcucNG5BMoP1lODBjpb7D6QJ7hhZjSzHLTmRYUm8IdGF2FfF5dq2lg73l0zaHclqKYzYN_ddLNq1eQ3pkVXNqzNJCA0c25nzyYSfZQcaEbxIdnl4fledx_cZzoIdxM/s400/HRA%20gif%20%281%29.gif" /></a></div>
<p>
<b>Collaborating to combat mobile fraud</b>
</p>
<p>
This enhanced fraud protection has undergone testing by the Singapore government and will be rolling out to Android devices with Google Play services.
</p>
<p>
“The fight against online scams is a dynamic one. As cybercriminals refine their methods, we must collaborate and innovate to stay ahead, “ said Mr Chua Kuan Seah, Deputy Chief Executive of CSA. “Through such partnerships with technology players like Google, we are constantly improving our anti-scam defenses to protect Singaporeans online and safeguard their digital assets.”
</p>
<p>
Together with CSA, we will be closely monitoring the results of the pilot program to assess its impact and make adjustments as needed. We will also support CSA by continuing to assist with malware detection and analysis, sharing malware insights and techniques, and creating user and developer education resources.
</p>
<p>
<b>How developers can prepare</b><br /><br />For developers distributing apps that may be affected by this pilot, please take the time to review the device <a href="https://developer.android.com/guide/topics/permissions/overview">permissions</a> your app is requesting and ensure you’re following <a href="https://developer.android.com/privacy-and-security/about">developer best practices</a>. Your app should only request permissions that the app needs to complete an action and ensure it does not violate the <a href="https://developers.google.com/android/play-protect/mobile-unwanted-software">Mobile Unwanted Software principles</a>. Always ensure that your app does not engage in behavior that could be considered potentially harmful or malware.
</p>
<p>
If you find that your app is affected by the app protection pilot you can refer to our <a href="https://developers.google.com/android/play-protect/warning-dev-guidance">updated developer guidance for Play Protect warnings</a> for tips on how to help fix potential issues with your app and instructions for filing an appeal if needed.
</p><p>Check out the video below to learn more.</p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="407" src="https://www.youtube.com/embed/DeEmqBydY24" width="489" youtube-src-id="DeEmqBydY24"></iframe></div><p><b>Our commitment to protecting Android users</b></p>
<p>
We believe industry collaboration is essential to protect users from mobile security threats and fraud. Piloting these new protections will help us stay ahead of new attacks and evolve our solutions to defeat scammers and their expanding fraud attempt. We have an unwavering commitment to protecting our users around the world and look forward to continuing to partner with governments, ecosystem partners and other stakeholders to improve user protections.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-89273045019462941942024-02-05T11:59:00.002-05:002024-02-06T21:30:31.880-05:00Improving Interoperability Between Rust and C++<span class="byline-author">Posted by Lars Bergstrom – Director, Android Platform Tools & Libraries and Chair of the Rust Foundation Board</span>
<p>
Back in 2021, we <a href="https://opensource.googleblog.com/2021/02/google-joins-rust-foundation.html">announced</a> that Google was joining the Rust Foundation. At the time, Rust was already in wide use across Android and other Google products. Our announcement emphasized our commitment to improving the security reviews of Rust code and its interoperability with C++ code. Rust is one of the strongest tools we have to address memory safety security issues. Since that announcement, industry leaders and <a href="https://www.cisa.gov/news-events/news/urgent-need-memory-safety-software-products">government agencies</a> have echoed our sentiment.
</p>
<p>
We are delighted to announce that Google has provided a grant of $1 million to the Rust Foundation to support efforts that will improve the ability of Rust code to interoperate with existing legacy C++ codebases. We’re also furthering our existing commitment to the open-source Rust community by aggregating and publishing <a href="https://opensource.googleblog.com/2023/05/open-sourcing-our-rust-crate-audits.html">audits for Rust crates</a> that we use in open-source Google projects. These contributions, along with our <a href="https://security.googleblog.com/2023/01/supporting-use-of-rust-in-chromium.html">previous interoperability contributions</a>, have us excited about the future of Rust.</p>
<blockquote><i>“Based on historical vulnerability density statistics, Rust has proactively prevented hundreds of vulnerabilities from impacting the Android ecosystem. This investment aims to expand the adoption of Rust across various components of the platform.”</i> <b><span style="color: #073763;"> – Dave Kleidermacher, Google Vice President of Engineering, Android Security & Privacy</span></b></blockquote><p></p>
<p>
While Google has seen the most significant growth in the use of Rust in Android, we’re continuing to grow its use across more applications, including clients and server hardware.
</p>
<blockquote> <i>“While Rust may not be suitable for all product applications, prioritizing seamless interoperability with C++ will accelerate wider community adoption, thereby aligning with the industry goals of improving memory safety.” </i><b><span style="color: #073763;">– Royal Hansen, Google Vice President of Safety & Security</span></b></blockquote><p></p>
<p>
The Rust tooling and ecosystem already support <a href="https://security.googleblog.com/2021/06/rustc-interop-in-android-platform.html">interoperability with Android</a> and with continued investment in tools like <a href="https://github.com/dtolnay/cxx">cxx</a>, <a href="https://github.com/google/autocxx">autocxx</a>, <a href="https://rust-lang.github.io/rust-bindgen/">bindgen</a>, <a href="https://github.com/eqrion/cbindgen">cbindgen</a>, <a href="https://github.com/rust-diplomat/diplomat">diplomat</a>, and <a href="https://github.com/google/crubit">crubit</a>, we are seeing regular improvements in the state of Rust interoperability with C++. As these improvements have continued, we’ve seen a reduction in the barriers to adoption and accelerated adoption of Rust. While that progress across the many tools continues, it is often only expanded incrementally to support the particular needs of a given project or company.
</p>
<p>
In order to accelerate both Rust adoption at Google as well as more broadly across the industry, we are eager to invest in and collaborate on any needed ABI changes, tooling and build system support, wrapper libraries, or other areas identified.
</p>
<p>
We are excited to support this work through the Rust Foundation’s Interop Initiative and in collaboration with the Rust project to ensure that any additions made are suitable and address the challenges of Rust adoption that projects using C++ face. Improving memory safety across the software industry is one of the key technology challenges of our time, and we invite others across the community and industry to join us in working together to secure the open source ecosystem for everyone.
</p>
<p>
Learn more about the Rust Foundation’s Interop Initiative by reading their recent <a href="https://rustfoundation.org/news/google-contributes-1m-to-rust-foundation-to-support-c-rust-interop-initiative/">announcement</a>.
</p>
<p></p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-83882966755291331042024-02-01T13:39:00.006-05:002024-02-01T13:40:22.956-05:00UN Cybercrime Treaty Could Endanger Web Security<span class="byline-author">Royal Hansen, Vice President of Privacy, Safety and Security Engineering</span><div><br /></div><div><span id="docs-internal-guid-246a9d66-7fff-3119-c5b8-87dcb74268e6"><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">This week, the United Nations </span><a href="https://www.unodc.org/unodc/en/cybercrime/ad_hoc_committee/home" style="text-decoration-line: none;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">convened</span></a><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> member states to continue its years-long negotiations on the UN Cybercrime Treaty, titled “</span><a href="https://documents-dds-ny.un.org/doc/UNDOC/GEN/V23/084/92/PDF/V2308492.pdf?OpenElement" style="text-decoration-line: none;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Countering the Use of Information and Communications Technologies for Criminal Purposes</span></a><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.” </span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As more aspects of our lives intersect with the digital sphere, law enforcement around the world has increasingly turned to electronic evidence to investigate and disrupt criminal activity. </span><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Google takes the threat of cybercrime very seriously, and dedicates significant resources to combating it. When governments send Google legal orders to disclose user data in connection with their investigations, we carefully review those orders to make sure they satisfy applicable laws, international norms, and Google’s policies. We also regularly report the number of these orders in our </span><a href="https://transparencyreport.google.com/user-data/overview?hl=en" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Transparency Report</span></a><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. </span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">To ensure that transnational legal demands are issued consistent with rule of law, we have long called for an </span><a href="https://blog.google/outreach-initiatives/public-policy/international-framework-digital-evidence/" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">international framework for digital evidence</span></a><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that includes robust due process protections, respects human rights (including the right to free expression), and aligns with existing international norms. This is particularly important in the case of transnational criminal investigations, where the legal protections in one jurisdiction may not align with those in others. </span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Such safeguards aren’t just important to ensuring free expression and human rights, they are also critical to protecting web security. Too often, as we know well from helping stand up the </span><a href="https://blog.google/technology/safety-security/new-initiatives-to-reduce-the-risk-of-vulnerabilities-and-protect-researchers/" style="text-decoration-line: none;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Security Researcher Legal Defense Fund</span></a><span face=""Google Sans Text", sans-serif" style="background-color: transparent; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, individuals working to advance cybersecurity for the public good end up facing criminal charges. The Cybercrime Treaty should not criminalize the work of legitimate cybersecurity researchers and penetration testers, which is designed to protect individual systems and the web as a whole. </span><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> </span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">UN Member States have an opportunity to strengthen global cybersecurity by adopting a treaty that encourages the criminalization of the most egregious and systemic activities — on which all parties can agree — while adopting a framework for sharing digital evidence that is transparent, grounded in the rule of law, based on pre-existing international frameworks like the Universal Declaration on Human Rights, and aligned with principles of necessity and proportionality. At the same time, Member States should avoid attempts to criminalize activities that raise significant freedom of expression issues, or that actually undercut the treaty’s goal of reducing cybercrime. That will require strengthening critical guardrails and protections. </span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="background-color: white; line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We urge Member States to heed </span><a href="https://docs.google.com/document/d/10iNCe3WUhex32Z-oFPbZoWhI4E3gtsO1n6KmxNGXA-4/edit" style="text-decoration-line: none;"><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">calls</span></a><span face=""Google Sans Text", sans-serif" style="background-color: transparent; color: #202124; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> from civil society groups to address critical gaps in the Treaty and revise the text to protect users and security professionals — not endanger the security of the web. </span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-2048568491937504292024-01-31T13:06:00.002-05:002024-02-13T13:59:43.512-05:00Scaling security with AI: from detection to solution<span class="byline-author">Dongge Liu and Oliver Chang, Google Open Source Security Team, Jan Nowakowski and Jan Keller, Machine Learning for Security Team</span><div><br /></div><div><span id="docs-internal-guid-003e8266-7fff-adc9-7a8a-c678192fa04d"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The AI world moves fast, so we’ve been hard at work keeping security apace with recent advancements. One of our approaches, in alignment with Google’s </span><a href="https://safety.google/cybersecurity-advancements/saif/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Secure AI Framework</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> (SAIF), is using AI itself to automate and streamline routine and manual security tasks, including fixing security bugs. Last year we wrote about our experiences </span><a href="https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug-hunting.html" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">using LLMs to expand vulnerability testing coverage</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, and we’re excited to share some updates. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Today, we’re </span><a href="https://github.com/google/oss-fuzz-gen" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">releasing our fuzzing framework</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> as a free, open source resource that researchers and developers can use to improve fuzzing’s bug-finding abilities. We’ll also show you how we’re using AI to speed up the bug patching process. By sharing these experiences, we hope to spark new ideas and drive innovation for a stronger ecosystem security.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Update: AI-powered vulnerability discovery</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Last August, we </span><a href="https://security.googleblog.com/2023/08/ai-powered-fuzzing-breaking-bug-hunting.html" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">announced</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> our framework to automate manual aspects of fuzz testing (“fuzzing”) that often hindered open source maintainers from fuzzing their projects effectively. We used LLMs to write project-specific code to boost fuzzing coverage and find more vulnerabilities. Our initial results on a subset of projects in our free </span><a href="https://google.github.io/oss-fuzz/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">OSS-Fuzz service</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> were very promising, with code coverage increased by 30% in one example. Since then, we’ve expanded our experiments to more than 300 OSS-Fuzz C/C++ projects, resulting in significant coverage gains across many of the project codebases. We’ve also improved our </span><a href="https://github.com/google/oss-fuzz-gen/blob/main/USAGE.md#Setup-Prompt-Templates" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">prompt generation</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and build pipelines, which has increased code line coverage by up to 29% in 160 projects. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><br /></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">How does that translate to tangible security improvements? So far, the expanded fuzzing coverage offered by LLM-generated improvements allowed OSS-Fuzz to discover two new vulnerabilities in </span><a href="https://github.com/DaveGamble/cJSON/issues/800" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">cJSON</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and </span><a href="https://github.com/libimobiledevice/libplist/issues/244" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">libplist</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, two widely used projects that had already been fuzzed for years. As always, we reported the vulnerabilities to the project maintainers for patching. Without the completely LLM-generated code, these two vulnerabilities could have remained undiscovered and unfixed indefinitely. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><br /></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">And more: AI-powered vulnerability </span><span face="Arial, sans-serif" style="font-size: 20pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">fixing</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Fuzzing is fantastic for finding bugs, but for security to improve, those bugs also need to be patched. It’s long been an industry-wide struggle to find the engineering hours needed to patch open bugs at the pace that they are uncovered, and triaging and fixing bugs is a significant manual toll on project maintainers. With continued improvements in using LLMs to find more bugs, we need to keep pace in creating similarly automated solutions to help fix those bugs. We recently </span><a href="https://research.google/pubs/pub52980/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">announced an experiment</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> doing exactly that: building an automated pipeline that intakes vulnerabilities (such as those caught by fuzzing), and prompts LLMs to generate fixes and test them before selecting the best for human review.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">This AI-powered patching approach resolved 15% of the targeted bugs, leading to significant time savings for engineers. The potential of this technology should apply to most or all categories throughout the software development process. We’re optimistic that this research marks a promising step towards harnessing AI to help ensure more secure and reliable software.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Try it out</span></h1><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Since we’ve now </span><a href="https://github.com/google/oss-fuzz-gen" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">open sourced our framework</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to automate manual aspects of fuzzing, any researcher or developer can experiment with their own prompts to test the effectiveness of fuzz targets generated by LLMs (including Google’s </span><a href="https://cloud.google.com/vertex-ai" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">VertexAI</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> or their own fine-tuned models) and measure the results against OSS-Fuzz C/C++ projects. We also hope to encourage research collaborations and to continue seeing </span><a href="https://arxiv.org/pdf/2312.17677.pdf" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">other work inspired by our approach</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, such as </span><a href="https://research.kudelskisecurity.com/2023/12/07/introducing-fuzzomatic-using-ai-to-automatically-fuzz-rust-projects-from-scratch/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Rust fuzz target generation</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><br /></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">If you’re interested in using LLMs to </span><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">patch</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> bugs, be sure to read </span><a href="https://research.google/pubs/pub52980/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">our paper</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> on building an AI-powered patching pipeline. You’ll find a summary of our own experiences, some unexpected data about LLM’s abilities to patch different types of bugs, and guidance for building pipelines in your own organizations. </span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-51049322385535866782024-01-30T12:00:00.002-05:002024-01-30T20:08:49.174-05:00Effortlessly upgrade to Passkeys on Pixel phones with Google Password Manager<span class="byline-author">Posted by Sherif Hanna, Group Product Manager, Pixel Security</span>
<br><p>
<strong><em>Helping Pixel owners upgrade to the easier, safer way to sign in</em></strong>
</p>
<p>
Your phone contains a lot of your personal information, from financial data to photos. Pixel phones are designed to help protect you and your data, and make security and privacy as easy as possible. This is why the Pixel team has been especially excited about <a href="https://blog.google/technology/safety-security/the-beginning-of-the-end-of-the-password/">passkeys</a>—the easier, safer alternative to passwords.
</p>
<p>
Passkeys are safer because they’re unique to each account, and are more resistant against online attacks such as phishing. They’re easier to use because there’s nothing for you to remember: when it’s time to sign in, using a passkey is as simple as unlocking your device with your face or fingerprint, or your PIN/pattern/password.
</p>
<p>
Google is working to accelerate passkey adoption. We’ve launched support for passkeys on Google platforms such as Android and Chrome, and recently we <a href="https://blog.google/technology/safety-security/passkeys-default-google-accounts/">announced</a> that we’re making passkeys a default option across personal Google Accounts. We’re also working with our partners across the industry to make passkeys available on more websites and apps.
</p>
<p>
Recently, we took things a step further. As part of last December’s <a href="https://blog.google/products/pixel/pixel-feature-drop-december-2023/">Pixel Feature Drop</a>, we introduced a new feature to Google Password Manager: passkey upgrades. With this new feature, Google Password Manager will let you discover which of your accounts support passkeys, and help you upgrade with just a few taps.
</p>
<p>
This new passkey upgrade experience is now available on Pixel phones (starting from Pixel 5a) as well as Pixel Tablet. Google Password manager will incorporate these updates for other platforms in the future.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyEgL75JXK4f7GsKD2Aje0r6OkMMRbNrfzebtITh5okEt8MD81sK1DElWSjTYx20KBzYL06VHIrNZ8Hv7f2w9yjaQdYmcFku5jm8lRI_FXB0HLJddoTv5e3gGNhddB1mc_WYWJAELEO8pqTwsnlY7GL03pbK0gDoO-fHAFmGhoGp5iv5FUw1C9oqhUhTAp/s1784/Hero%20image%20-%202%20panels.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="400" data-original-height="1784" data-original-width="1726" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyEgL75JXK4f7GsKD2Aje0r6OkMMRbNrfzebtITh5okEt8MD81sK1DElWSjTYx20KBzYL06VHIrNZ8Hv7f2w9yjaQdYmcFku5jm8lRI_FXB0HLJddoTv5e3gGNhddB1mc_WYWJAELEO8pqTwsnlY7GL03pbK0gDoO-fHAFmGhoGp5iv5FUw1C9oqhUhTAp/s400/Hero%20image%20-%202%20panels.png"/></a></div>
Best of all, today we’re happy to announce that we’ve teamed up with Adobe, Best Buy, DocuSign, eBay, Kayak, Money Forward, Nintendo, PayPal, Uber, Yahoo! Japan—and soon, TikTok as well, to help bring you this easy passkey upgrade experience and usher you into the passwordless future.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlZR5BkKPPaJVeIUwv2Wav10JnM4NN4Bizbm1XNI-nRfGnlWcVonpt_L6DJPQfvuk_E0zoiW7Kc9dcYsNjEJVbIRKf4sdSjeVNQr_8IJJanA5CZRIYerXK906UNKsWJOgdruVbq6OnXqXjkvnnpsgN7AeuCHitASmm6ZozRcD_nRMuUADO4Rhf3nC4wBvT/s1600/PKW%20logos.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="2097" data-original-width="3219" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlZR5BkKPPaJVeIUwv2Wav10JnM4NN4Bizbm1XNI-nRfGnlWcVonpt_L6DJPQfvuk_E0zoiW7Kc9dcYsNjEJVbIRKf4sdSjeVNQr_8IJJanA5CZRIYerXK906UNKsWJOgdruVbq6OnXqXjkvnnpsgN7AeuCHitASmm6ZozRcD_nRMuUADO4Rhf3nC4wBvT/s1600/PKW%20logos.png"/></a></div>
If you have an account with one of these early launch partners, Google Password Manager on Pixel will helpfully guide you to the exact location on the partner’s website or app where you can upgrade to a passkey. There’s no need to manually hunt for the option in account settings.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLkrRFAHgO_vZ1VirGIqJdNz-F7vKZuWouzZV7tAvU02AMnYo6jNf-GXzKIeeWjQ52-xmNkdTBrOIW8ceE-BVLy9_1RBEFoYir7YImH81hyphenhyphenVw8duwRPiQiJzziIILqb6DsfEs5qgzeLfWfl4o801BRbjVd1buwaTAZ2Tx6fHLvtPre2itxA-lS30PFe5jN/s2656/Hero%20image%20-%203%20panels.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" width="600" data-original-height="1787" data-original-width="2656" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLkrRFAHgO_vZ1VirGIqJdNz-F7vKZuWouzZV7tAvU02AMnYo6jNf-GXzKIeeWjQ52-xmNkdTBrOIW8ceE-BVLy9_1RBEFoYir7YImH81hyphenhyphenVw8duwRPiQiJzziIILqb6DsfEs5qgzeLfWfl4o801BRbjVd1buwaTAZ2Tx6fHLvtPre2itxA-lS30PFe5jN/s600/Hero%20image%20-%203%20panels.png"/></a></div>
And because the technology that makes this possible is <a href="https://developers.google.com/identity/passkeys/developer-guides/upgrades">open</a>, any website or app, as well as any other password manager, can leverage it to help their users upgrade to passkeys for supporting accounts. It’s all part of Google’s commitment to help make signing in easier and safer.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-13148394332391356732024-01-11T12:00:00.034-05:002024-01-11T14:18:14.521-05:00MiraclePtr: protecting users from use-after-free vulnerabilities on more platforms<span class="byline-author">Posted by Keishi Hattori, Sergei Glazunov, Bartek Nowierski on behalf of the MiraclePtr team</span><p>
Welcome back to our latest update on MiraclePtr, our project to protect against use-after-free vulnerabilities in Google Chrome. If you need a refresher, you can read our <a href="https://security.googleblog.com/2022/09/use-after-freedom-miracleptr.html">previous blog post</a> detailing MiraclePtr and its objectives.
</p>
<h1 style="text-align: left;">More platforms</h1>
<p>
We are thrilled to announce that since our last update, we have successfully enabled MiraclePtr for more platforms and processes:
</p>
<ul>
<li>In June 2022, we enabled MiraclePtr for the browser process on Windows and Android.
</li><li>In September 2022, we expanded its coverage to include all processes except renderer processes.
</li><li>In June 2023, we enabled MiraclePtr for ChromeOS, macOS, and Linux.
</li>
</ul>
<p>
Furthermore, we have changed <a href="https://chromium.googlesource.com/chromium/src/+/main/docs/security/severity-guidelines.md#TOC-MiraclePtr">security guidelines</a> to downgrade MiraclePtr-protected issues by one severity level!
</p>
<h1 style="text-align: left;">Evaluating Security Impact</h1>
<p>
First let’s focus on its security impact. Our analysis is based on two primary information sources: incoming vulnerability reports and crash reports from user devices. Let's take a closer look at each of these sources and how they inform our understanding of MiraclePtr's effectiveness.
</p>
<h3 style="text-align: left;">Bug reports</h3>
<p>
Chrome vulnerability reports come from various sources, such as:
</p>
<ul>
<li><a href="https://bughunters.google.com/about/rules/5745167867576320/chrome-vulnerability-reward-program-rules">Chrome Vulnerability Reward Program</a> participants,
</li><li>our fuzzing infrastructure,
</li><li>internal and external teams investigating security incidents.
</li>
</ul>
<p>
For the purposes of this analysis, we focus on vulnerabilities that affect platforms where MiraclePtr was enabled at the time the issues were reported. We also exclude bugs that occur inside a sandboxed renderer process. Since the initial launch of MiraclePtr in 2022, we have received 168 use-after-free reports matching our criteria.
</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQTJTVefnA6pPD_rGBgiSUGK0uGcMjowzdIHw89xIa-cdaRybspLntIMcGQCBAJvhbxmS7sZCE9UThhQ-2SRgtKJ100BKklVX8WIaj-ovJ5PBau1TGypbxSIYyMfxcu0F7gf-8f6qoUQSfbw2E2kYp5ErGsDww53EbNi99mfeD_vqWu9xPFJAB0cRiBgMC/s679/Screenshot%202024-01-10%2011.40.12%20AM.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="396" data-original-width="679" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQTJTVefnA6pPD_rGBgiSUGK0uGcMjowzdIHw89xIa-cdaRybspLntIMcGQCBAJvhbxmS7sZCE9UThhQ-2SRgtKJ100BKklVX8WIaj-ovJ5PBau1TGypbxSIYyMfxcu0F7gf-8f6qoUQSfbw2E2kYp5ErGsDww53EbNi99mfeD_vqWu9xPFJAB0cRiBgMC/s16000/Screenshot%202024-01-10%2011.40.12%20AM.png" /></a></div>
<p>
What does the data tell us? MiraclePtr effectively <strong>mitigated 57% of these use-after-free vulnerabilities in privileged processes</strong>, exceeding our initial estimate of 50%. Reaching this level of effectiveness, however, required additional work. For instance, we not only rewrote class fields to use MiraclePtr, as discussed in the previous post, but also added MiraclePtr support for bound function arguments, such as <code>Unretained</code> pointers. These pointers have been a significant source of use-after-frees in Chrome, and the additional protection allowed us to mitigate 39 more issues.
</p>
<p>
Moreover, these vulnerability reports enable us to pinpoint areas needing improvement. We're actively working on adding support for select third-party libraries that have been a source of use-after-free bugs, as well as developing a more advanced rewriter tool that can handle transformations like converting <code>std::vector<T*></code> into <code>std::vector<raw_ptr<T>></code>. We've also made several smaller fixes, such as extending the lifetime of the task state object to cover several issues in the “<code>this</code> pointer” category.
</p>
<h3 style="text-align: left;">Crash reports</h3>
<p>
Crash reports offer a different perspective on MiraclePtr's effectiveness. As explained in the previous blog post, when an allocation is quarantined, its contents are overwritten with a special bit pattern. If the allocation is used later, the pattern will often be interpreted as an invalid memory address, causing a crash when the process attempts to access memory at that address. Since the dereferenced address remains within a small, predictable memory range, we can distinguish MiraclePtr crashes from other crashes.
</p>
<p>
Although this approach has its limitations — such as not being able to obtain stack traces from allocation and deallocation times like <a href="https://github.com/google/sanitizers/wiki/AddressSanitizer">AddressSanitizer</a> does — it has enabled us to detect and fix vulnerabilities. Last year, <a href="https://bugs.chromium.org/p/chromium/issues/list?q=opened%3E2021-12-31%20opened%3C2023-01-01%20Security_Impact%3DExtended%2CStable%20Security_Severity%3DCritical%20status%3Afixed&can=1">six</a> critical severity vulnerabilities were identified in the default setup of Chrome Stable, the version most people use. Impressively, five of the six were discovered while investigating MiraclePtr crash reports! One particularly interesting example is <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=1340253">CVE-2022-3038</a>. The issue was discovered through MiraclePtr crash reports and fixed in Chrome 105. Several months later, Google's Threat Analysis Group <a href="https://blog.google/threat-analysis-group/spyware-vendors-use-0-days-and-n-days-against-popular-platforms/">discovered</a> an exploit for that vulnerability used in the wild against clients of a different Chromium-based browser that hadn’t shipped the fix yet.
</p>
<p>
To further enhance our crash analysis capabilities, we've recently launched <a href="https://docs.google.com/document/d/1xfGa_IMtFZiQ3beOmkncEafODwn4U90ZyL4NfPaAtDY/edit?pli=1&resourcekey=0-89BZl1SVILB6ylOHula0IA#heading=h.ug7k4kk8zc84">an experimental feature</a> that allows us to collect additional information for MiraclePtr crashes, including stack traces. This effectively shortens the average crash report investigation time.
</p>
<h1 style="text-align: left;">Performance</h1>
<p>
MiraclePtr enables us to have robust protection against use-after-free bug exploits, but there is a performance cost associated with it. Therefore, we have conducted experiments on each platform where we have shipped MiraclePtr, which we used in our decision-making process.
</p>
<p>
The main cost of MiraclePtr is memory. Specifically, the memory usage of the browser process increased by 5.5-8% on desktop platforms and approximately 2% on Android. Yet, when examining the holistic memory usage across all processes, the impact remains within a moderate 1-3% range to lower percentiles only.
</p>
<p>
The main cause of the additional memory usage is the extra size to allocate the reference count. One might think that adding 4 bytes to each allocation wouldn’t be a big deal. However, there are many small allocations in Chrome, so even the 4B overhead is not negligible. Moreover, PartitionAlloc also uses pre-defined allocation bucket sizes, so this extra 4B pushes certain allocations (particularly power-of-2 sized) into a larger bucket, e.g. 4096B → 5120B.
</p>
<p>
We also considered the performance cost. We verified that there were no regressions to the majority of our top-level performance metrics, including all of the page load metrics, like Largest Contentful Paint, First Contentful Paint and Cumulative Layout Shift. We did find a few regressions, such as a 10% increase in the 99th percentile of the browser process <a href="https://docs.google.com/document/d/1vDSGFvJblh7yJ3U3RVB_7qZLubyfTbQdQjuN1GoUNkc/edit">main thread contention metric</a>, a 1.5% regression in First Input Delay on ChromeOS, and a 1.5% regression in tab startup time on Android. The main thread contention metric tries to estimate how often a user input can be delayed and so for example on Windows this was a change from 1.6% to 1.7% at the 99th percentile only. These are all minor regressions. There has been zero change in daily active usage, and we do not anticipate these regressions to have any noticeable impact on users.
</p>
<h1 style="text-align: left;">Conclusion</h1>
<p>
In summary, MiraclePtr has proven to be effective in mitigating use-after-free vulnerabilities and enhancing the overall security of the Chrome browser. While there are performance costs associated with the implementation of MiraclePtr, our analysis suggests that the benefits in terms of security improvements far outweigh these. We are committed to continually refining and expanding the feature to cover more areas. For example we are working to add coverage to third-party libraries used by the GPU process, and we plan to enable BRP on the renderer process. By sharing our findings and experiences, we hope to contribute to the broader conversation surrounding browser security and inspire further innovation in this crucial area.
</p>Googlehttp://www.blogger.com/profile/11822708545141062574noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-22678947344900983632023-12-12T12:00:00.000-05:002023-12-12T12:00:09.520-05:00Hardening cellular basebands in Android<span class="byline-author">Posted by Ivan Lozano and Roger Piqueras Jover</span>
<p>
Android’s defense-in-depth strategy applies not only to the Android OS running on the Application Processor (AP) but also the firmware that runs on devices. We particularly <a href="https://security.googleblog.com/2023/02/hardening-firmware-across-android.html">prioritize hardening the cellular baseband</a> given its unique combination of running in an elevated privilege and parsing untrusted inputs that are remotely delivered into the device.
</p>
<p>
This post covers how to use two high-value sanitizers which can prevent specific classes of vulnerabilities found within the baseband. They are architecture agnostic, suitable for bare-metal deployment, and should be enabled in existing C/C++ code bases to mitigate unknown vulnerabilities. Beyond security, addressing the issues uncovered by these sanitizers improves code health and overall stability, reducing resources spent addressing bugs in the future.
</p>
<h1>An increasingly popular attack surface</h1>
<p>
As we <a href="https://security.googleblog.com/2023/02/hardening-firmware-across-android.html">outlined previously</a>, security research focused on the baseband has highlighted a consistent lack of exploit mitigations in firmware. Baseband Remote Code Execution (RCE) exploits <a href="https://www.zerodium.com/images/zerodium_prices_mobiles.png">have their own categorization</a> in well-known third-party marketplaces with a relatively low payout. This suggests baseband bugs may potentially be abundant and/or not too complex to find and exploit, and their prominent inclusion in the marketplace demonstrates that they are useful.
</p>
<p>
Baseband security and exploitation has been a <a href="https://www.usenix.org/system/files/conference/woot12/woot12-final24.pdf">recurring</a> <a href="https://www.blackhat.com/docs/us-14/materials/us-14-Lindh-Attacking-Mobile-Broadband-Modems-Like-A-Criminal-Would.pdf">theme</a> in <a href="https://vimeo.com/showcase/4562410/video/214013463">security</a> <a href="https://speakerdeck.com/marcograss/exploitation-of-a-modern-smartphone-baseband">conferences</a> <a href="https://i.blackhat.com/USA21/Wednesday-Handouts/us-21-Over-The-Air-Baseband-Exploit-Gaining-Remote-Code-Execution-On-5G-Smartphones.pdf">for the last decade</a>. Researchers have also made a dent in this area in <a href="https://www.forbes.com/sites/daveywinder/2019/11/10/samsung-galaxy-s10-hacked-twice/?sh=54b24bb25bd7">well-known exploitation contests</a>. Most recently, this area has become prominent enough that it is common to find practical baseband exploitation <a href="https://milano.securitybsides.it/baseband-exploitation.html">trainings</a> in <a href="https://hardwear.io/usa-2023/training/reverse-engineering-emulation-dynamic-testing-cellular-baseband-firmware.php">top</a> <a href="https://www.offensivecon.org/trainings/2023/baseband-exploitation.html">security</a> <a href="https://www.hexacon.fr/trainer/ribeiro_burke/">conferences</a>.
</p>
<p>
Acknowledging this trend, combined with the severity and apparent abundance of these vulnerabilities, last year we introduced updates to the <a href="https://source.android.com/docs/security/overview/updates-resources#severity">severity guidelines</a> of Android’s Vulnerability Rewards Program (VRP). For example, we consider vulnerabilities allowing Remote Code Execution (RCE) in the cellular baseband to be of CRITICAL severity.
</p>
<h1>Mitigating Vulnerability Root Causes with Sanitizers</h1>
<p>
Common classes of vulnerabilities can be mitigated through the use of sanitizers provided by <a href="https://clang.llvm.org/">Clang-based toolchains</a>. These sanitizers insert runtime checks against common classes of vulnerabilities. GCC-based toolchains may also provide some level of support for these flags as well, but will not be considered further in this post. We encourage you to check your toolchain’s documentation.
</p>
<p>
Two sanitizers included in <a href="https://source.android.com/docs/security/test/ubsan">Undefined Behavior Sanitizer</a> (UBSan) will be our focus – <a href="https://source.android.com/docs/security/test/intsan">Integer Overflow Sanitizer</a> (IntSan) and <a href="https://source.android.com/docs/security/test/bounds-sanitizer">BoundsSanitizer</a> (BoundSan). These have been <a href="https://android-developers.googleblog.com/2016/05/hardening-media-stack.html">widely deployed</a> in <a href="https://android-developers.googleblog.com/2018/06/compiler-based-security-mitigations-in.html">Android userspace</a> <a href="https://android-developers.googleblog.com/2019/05/queue-hardening-enhancements.html">for years</a> following <a href="https://security.googleblog.com/2021/01/data-driven-security-hardening-in.html">a data-driven approach</a>. These two are well suited for bare-metal environments such as the baseband since they do not require support from the OS or specific architecture features, and so are generally supported for all Clang targets.
</p>
<h2>Integer Overflow Sanitizer (IntSan)</h2>
<p>
IntSan causes signed and unsigned integer overflows to abort execution unless the overflow is made explicit. While unsigned integer overflows are technically defined behavior, it can often lead to unintentional behavior and vulnerabilities – especially when they’re used to index into arrays.
</p>
<p>
As both intentional and unintentional overflows are likely present in most code bases, IntSan may require refactoring and annotating the code base to prevent intentional or benign overflows from trapping (which we consider a false positive for our purposes). Overflows which need to be addressed can be uncovered via testing (see the Deploying Sanitizers section)
</p>
<h2>BoundsSanitizer (BoundSan)</h2>
<p>
BoundSan inserts instrumentation to perform bounds checks around some array accesses. These checks are only added if the compiler cannot prove at compile time that the access will be safe and if the size of the array will be known at runtime, so that it can be checked against. Note that this will not cover all array accesses as the size of the array may not be known at runtime, such as function arguments which are arrays.
</p>
<p>
As long as the code is correctly written C/C++, BoundSan should produce no false positives. Any violations discovered when first enabling BoundSan is at least a bug, if not a vulnerability. Resolving even those which aren’t exploitable can greatly improve stability and code quality.
</p>
<h2>Modernize your toolchains</h2>
<p>
Adopting modern mitigations also means adopting (and maintaining) modern toolchains. The benefits of this go beyond utilizing sanitizers however. Maintaining an old toolchain is not free and entails hidden opportunity costs. Toolchains contain bugs which are addressed in <a href="https://releases.llvm.org/17.0.1/tools/clang/docs/ReleaseNotes.html#bug-fixes-in-this-version">subsequent releases</a>. Newer toolchains bring new <a href="https://releases.llvm.org/17.0.1/tools/clang/docs/ReleaseNotes.html#bug-fixes-in-this-version:~:text=Improved%20code%20generation,MS%20C%2B%2B%20ABI.">performance optimizations</a>, valuable in the highly constrained bare-metal environment that basebands operate in. <a href="https://nvd.nist.gov/vuln/detail/CVE-2018-12886">Security</a> <a href="https://nvd.nist.gov/vuln/detail/CVE-2023-4039">issues</a> can even exist in the generated code of out-of-date compilers.
</p>
<p>
Maintaining a modern up-to-date toolchain for the baseband entails some costs in terms of maintenance, especially at first if the toolchain is particularly old, but over time the benefits, as outlined above, outweigh the costs.
</p>
<h1>Where to apply sanitizers</h1>
<p>
Both BoundSan and IntSan have a measurable performance overhead. Although we were able to significantly reduce this overhead in the past (<a href="https://security.googleblog.com/2019/05/queue-hardening-enhancements.html">for example to less than 1% in media codecs</a>), even very small increases in CPU load can have a substantial impact in some environments.
</p>
<p>
Enabling sanitizers over the entire codebase provides the most benefit, but enabling them in security-critical attack surfaces can serve as a first step in an incremental deployment. For example:
</p>
<ul>
<li>Functions parsing messages delivered over the air in 2G, 3G, 4G, and 5G (especially functions handling pre-authentication messages that can be injected with a false/malicious base station)
<li>Libraries encoding/decoding complex formats (e.g. ASN.1, XML, DNS, etc…)
<li>IMS, TCP and IP stacks
<li>Messaging functions (SMS, MMS)
</li>
</ul>
<p>
In the particular case of 2G, the best strategy is to disable the stack altogether by supporting Android’s “<a href="https://source.android.com/docs/setup/about/android-12-release#2g-toggle">2G toggle</a>”. However, 2G is still a necessary mobile access technology in certain parts of the world and some users might need to have this legacy protocol enabled.
</p>
<h1>Deploying Sanitizers</h1>
<p>
Having a clear plan for deployment of sanitizers saves a lot of time and effort. We think of the deployment process as having three stages:
</p>
<ul>
<li>Detecting (and fixing) violations
<li>Measuring and reducing overhead
<li>Soaking in pre-production
</li>
</ul>
<p>
We also introduce two modes in which sanitizers should be run: <strong>diagnostics mode</strong> and <strong>trapping mode</strong>. These will be discussed in the following sections, but briefly: diagnostics mode recovers from violations and provides valuable debug information, while trapping mode actively mitigates vulnerabilities by trapping execution on violations.
</p>
<h2>Detecting (and Fixing) Violations</h2>
<p>
To successfully ship these sanitizers, any benign integer overflows must be made explicit and accidental out-of-bounds accesses must be addressed. These will have to be uncovered through testing. The higher the code coverage your tests provide, the more issues you can uncover at this stage and the easier deployment will be later on.
</p>
<p>
To diagnose violations uncovered in testing, sanitizers can emit calls to runtime handlers with debug information such as the file, line number, and values leading to the violation. Sanitizers can optionally continue execution after a violation has occurred, allowing multiple violations to be discovered in a single test run. We refer to using the sanitizers in this way as running them in “<strong>diagnostics mode</strong>”. Diagnostics mode is not intended for production as it provides no security benefits and adds high overhead.
</p>
<p>
Diagnostics mode for the sanitizers can be set using the following flags:
</p>
<pre class="prettyprint">-fsanitize=signed-integer-overflow,unsigned-integer-overflow,bounds -fsanitize-recover=all</pre>
<p>
Since Clang does not provide a UBSan runtime for bare-metal targets, a runtime will need to be defined and provided at link time:
</p>
<pre class="prettyprint">// integer overflow handlers
__ubsan_handle_add_overflow(OverflowData *data, ValueHandle lhs, ValueHandle rhs)
__ubsan_handle_sub_overflow(OverflowData *data, ValueHandle lhs, ValueHandle rhs)
__ubsan_handle_mul_overflow(OverflowData *data, ValueHandle lhs, ValueHandle rhs)
__ubsan_handle_divrem_overflow(OverflowData *data, ValueHandle lhs, ValueHandle rhs)
__ubsan_handle_negate_overflow(OverflowData *data, ValueHandle old_val)
// boundsan handler
__ubsan_handle_out_of_bounds_overflow(OverflowData *data, ValueHandle old_val)</pre>
<p>
As an example, see the <a href="https://cs.android.com/android-llvm/toolchain/llvm-project/+/master:compiler-rt/lib/ubsan/ubsan_handlers.cc">default Clang implementation</a>; the Linux Kernels <a href="https://cs.android.com/android/kernel/superproject/+/common-android-mainline:common/lib/ubsan.c;l=335?">implementation</a> may also be illustrative.
</p>
<p>
With the runtime defined, enable the sanitizer over the entire baseband codebase and run all available tests to uncover and address any violations. Vulnerabilities should be patched, and overflows should either be refactored or made explicit through the use of <a href="https://clang.llvm.org/docs/LanguageExtensions.html#checked-arithmetic-builtins">checked arithmetic builtins</a> which do not trigger sanitizer violations. Certain functions which are expected to have intentional overflows (such as cryptographic functions) can be preemptively excluded from sanitization (see next section).
</p>
<p>
Aside from uncovering security vulnerabilities, this stage is highly effective at uncovering code quality and stability bugs that could result in instability on user devices.
</p>
<p>
Once violations have been addressed and tests are no longer uncovering new violations, the next stage can begin.
</p>
<h2>Measuring and Reducing Overhead</h2>
<p>
Once shallow violations have been addressed, benchmarks can be run and the overhead from the sanitizers (performance, code size, memory footprint) can be measured.
</p>
<p>
Measuring overhead must be done using production flags – namely <strong>“trapping mode”</strong>, where violations cause execution to abort. The diagnostics runtime used in the first stage carries significant overhead and is not indicative of the actual performance sanitizer overhead.
</p>
<p>
Trapping mode can be enabled using the following flags:
</p>
<pre class="prettyprint">-fsanitize=signed-integer-overflow,unsigned-integer-overflow,bounds -fsanitize-trap=all</pre>
<p>
Most of the overhead is likely due to a small handful of “hot functions”, for example those with tight long-running loops. Fine-grained per-function performance metrics (similar to what <a href="https://android.googlesource.com/platform/system/extras/+/master/simpleperf/doc/README.md">Simpleperf</a> provides for Android), allows comparing metrics before and after sanitizers and provides the easiest means to identify hot functions. These functions can either be refactored or, after manual inspection to verify that they are safe, have sanitization disabled.
</p>
<p>
Sanitizers can be disabled either <a href="https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#disabling-instrumentation-with-attribute-no-sanitize-undefined">inline in the source</a> or through the use of <a href="https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#suppressing-errors-in-recompiled-code-ignorelist">ignorelists</a> and the -fsanitize-ignorelist flag.
</p>
<p>
The physical layer code, with its extremely tight performance margins and lower chance of exploitable vulnerabilities, may be a good candidate to disable sanitization wholesale if initial performance seems prohibitive.
</p>
<h2>Soaking in Pre-production</h2>
<p>
With overhead minimized and shallow bugs resolved, the final stage is enabling the sanitizers in trapping mode to mitigate vulnerabilities.
</p>
<p>
We strongly recommend a long period of internal soak in pre-production with test populations to uncover any remaining violations not discovered in testing. The more thorough the test coverage and length of the soak period, the less risk there will be from undiscovered violations.
</p>
<p>
As above, the configuration for trapping mode is as follows:
</p>
<pre class="prettyprint">-fsanitize=signed-integer-overflow,unsigned-integer-overflow,bounds -fsanitize-trap=all</pre>
<p>
Having infrastructure in place to collect bug reports which result from any undiscovered violations can help minimize the risk they present.
</p>
<h1>Transitioning to Memory Safe Languages</h1>
<p>
The benefits from deploying sanitizers in your existing code base are tangible, however ultimately they address only the lowest hanging fruit and will not result in a code base free of vulnerabilities. Other classes of memory safety vulnerabilities remain unaddressed by these sanitizers. A longer term solution is to begin transitioning <em>today</em> to memory-safe languages such as Rust.
</p>
<p>
<a href="https://security.googleblog.com/2023/10/bare-metal-rust-in-android.html">Rust is ready for bare-metal environments</a> such as the baseband, and we are already using it in other bare-metal components in Android. There is no need to rewrite everything in Rust, as Rust provides a strong C FFI support and easily interfaces with existing C codebases. <a href="https://security.googleblog.com/2021/04/rust-in-android-platform.html#:~:text=But%20what%20about%20all%20that%20existing%20C%2B%2B%3F">Just writing new code in Rust</a> can rapidly <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">reduce the number of memory safety vulnerabilities</a>. Rewrites should be limited/prioritized only for the most critical components, such as complex parsers handling untrusted data.
</p>
<p>
The Android team has developed a <a href="https://google.github.io/comprehensive-rust/">Rust training</a> meant to help experienced developers quickly ramp up Rust fundamentals. An entire day for <a href="https://google.github.io/comprehensive-rust/bare-metal.html">bare-metal Rust</a> is included, and the course has been translated to a number of different languages.
</p>
<p>
While the Rust compiler may not explicitly support your bare-metal target, because it is a front-end for LLVM, any target supported by LLVM can be supported in Rust through <a href="https://docs.rust-embedded.org/embedonomicon/custom-target.html">custom target definitions</a>.
</p>
<h1>Raising the Bar</h1>
<p>
As the high-level operating system becomes a more difficult target for attackers to successfully exploit, we expect that lower level components such as the baseband will attract more attention. By using modern toolchains and deploying exploit mitigation technologies, the bar for attacking the baseband can be raised as well. If you have any questions, let us know – we’re here to help!
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-86503847931320952212023-11-29T12:00:00.000-05:002023-11-29T12:00:03.168-05:00Improving Text Classification Resilience and Efficiency with RETVec<span class="byline-author">Elie Bursztein, Cybersecurity & AI Research Director, and Marina Zhang, Software Engineer</span><div><br /></div><div><br /></div><div><span id="docs-internal-guid-68f94493-7fff-a5c1-b2f4-2d2a96381e67"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Systems such as Gmail, YouTube and Google Play rely on text classification models to identify harmful content including phishing attacks, inappropriate comments, and scams. These types of texts are harder for machine learning models to classify because bad actors rely on adversarial text manipulations to actively attempt to evade the classifiers. For example, they will use homoglyphs, invisible characters, and keyword stuffing to bypass defenses. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">To help make text classifiers more robust and efficient, we’ve developed a novel, multilingual text vectorizer called </span><a href="https://github.com/google-research/retvec" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">RETVec</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> (Resilient & Efficient Text Vectorizer) that helps models achieve state-of-the-art classification performance and drastically reduces computational cost. Today, we’re sharing how RETVec has been used to help protect Gmail inboxes.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">Strengthening the Gmail Spam Classifier with RETVec</span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj85aSI-fhoVHsc75YdbNCdGSAJpLBXS6nqu06kMZc2GbyigrtgAuA-v8rfQxSMnDi9rCW5ULLqCVfg7iGbf7fovlQCPFUefWoRzzr89tnUTIUoHY4aM-Ts6Zg883MlXzIwlygKfTh7AaEtLrX7yEq-4LANgjeotpBRw3jsjiz6m6TNxt3XEp46DQfTh9g_/s1226/Screenshot%202023-11-29%20at%208.10.58%E2%80%AFAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="570" data-original-width="1226" height="253" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj85aSI-fhoVHsc75YdbNCdGSAJpLBXS6nqu06kMZc2GbyigrtgAuA-v8rfQxSMnDi9rCW5ULLqCVfg7iGbf7fovlQCPFUefWoRzzr89tnUTIUoHY4aM-Ts6Zg883MlXzIwlygKfTh7AaEtLrX7yEq-4LANgjeotpBRw3jsjiz6m6TNxt3XEp46DQfTh9g_/w543-h253/Screenshot%202023-11-29%20at%208.10.58%E2%80%AFAM.png" width="543" /></a></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Figure 1. RETVec-based Gmail Spam filter improvements.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Over the past year, we battle-tested RETVec extensively inside Google to evaluate its usefulness and found it to be highly effective for security and anti-abuse applications. In particular, replacing the Gmail spam classifier’s previous text vectorizer with RETVec allowed us to improve the spam detection rate over the baseline by 38% and reduce the false positive rate by 19.4%. Additionally, using RETVec reduced the TPU usage of the model by 83%, making the RETVec deployment one of the largest defense upgrades in recent years. RETVec achieves these improvements by sporting a very lightweight word embedding model (~200k parameters), allowing us to reduce the Transformer model’s size at equal or better performance, and having the ability to split the computation between the host and TPU in a network and memory efficient manner.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">RETVec Benefits</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">RETVec achieves these improvements by combining a novel, highly-compact character encoder, an augmentation-driven training regime, and the use of </span><a href="https://github.com/tensorflow/similarity" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">metric learning</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. The architecture details and benchmark evaluations are available in our </span><a href="https://arxiv.org/abs/2302.09207" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">NeurIPS 2023 paper</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and we </span><a href="https://github.com/google-research/retvec" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">open-source RETVec on Github.</span></a></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Due to its novel architecture, RETVec works out-of-the-box on every language and all UTF-8 characters without the need for text preprocessing, making it the ideal candidate for on-device, web, and large-scale text classification deployments. Models trained with RETVec exhibit faster inference speed due to its compact representation. Having smaller models reduces computational costs and decreases latency, which is critical for large-scale applications and on-device models.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVhNMiRcAxN0YEXkim8_aLlpT6YAM5PEDgcWT4LFb7EKibBVLK9cAbSGDffn7Hj7_KUo1JbjVHLOxS7YnQfdESFaDP2a2MpSm3dJfNnHL3-hcRizE_K-2QNvGIqlollmZT5qW1sI5u3Bxr7ZaP-HB2EkmRlFG9-IP-ew94-2UnDuQu5HrxU5HJVRVUIBOg/s1386/Screenshot%202023-11-29%20at%208.16.22%E2%80%AFAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="354" data-original-width="1386" height="165" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhVhNMiRcAxN0YEXkim8_aLlpT6YAM5PEDgcWT4LFb7EKibBVLK9cAbSGDffn7Hj7_KUo1JbjVHLOxS7YnQfdESFaDP2a2MpSm3dJfNnHL3-hcRizE_K-2QNvGIqlollmZT5qW1sI5u3Bxr7ZaP-HB2EkmRlFG9-IP-ew94-2UnDuQu5HrxU5HJVRVUIBOg/w644-h165/Screenshot%202023-11-29%20at%208.16.22%E2%80%AFAM.png" width="644" /></a></div><div class="separator" style="clear: both; text-align: center;"><br /></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Figure 1. RETVec architecture diagram.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Models trained with RETVec can be </span><a href="https://github.com/google-research/retvec/blob/main/notebooks/tf_lite_retvec.ipynb" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">seamlessly converted to TFLite</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> for mobile and edge devices, as a result of a native implementation in TensorFlow Text. For web application model deployment, we provide a TensorflowJS layer implementation that is available on Github and you can check out a </span><a href="https://google-research.github.io/retvec/emotion_demo" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">demo web page</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> running a RETVec-based model.</span></p><br /><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihY8WjRfg1Pgm0pqYCyEb7NaHYcvNLCz_E1WMi-ls4dl7WsvJDryD2U79VHQFXWpTJ3YR_UMXBj9XMXT114hXZAaUMFABbSKTC0fR2PUiWWsbvpPTbCxQUhlW3fo4umhrXGxjFrwrCecAqCcEL-upmY_QqDpUt-8d6qx5dS8lEOpYeREFmJgjoblx4-dtD/s1332/Screenshot%202023-11-29%20at%208.15.29%E2%80%AFAM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="790" data-original-width="1332" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihY8WjRfg1Pgm0pqYCyEb7NaHYcvNLCz_E1WMi-ls4dl7WsvJDryD2U79VHQFXWpTJ3YR_UMXBj9XMXT114hXZAaUMFABbSKTC0fR2PUiWWsbvpPTbCxQUhlW3fo4umhrXGxjFrwrCecAqCcEL-upmY_QqDpUt-8d6qx5dS8lEOpYeREFmJgjoblx4-dtD/w526-h312/Screenshot%202023-11-29%20at%208.15.29%E2%80%AFAM.png" width="526" /></a></div><br /><div class="separator" style="clear: both; text-align: center;"><br /></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Figure 2. Typo resilience of text classification models trained from scratch using different vectorizers.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">RETVec is a novel </span><a href="https://github.com/google-research/retvec" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">open-source text vectorizer</span></a><span style="font-family: Roboto, sans-serif; font-size: 10pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> </span><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">that allows you to build more resilient and efficient server-side and on-device text classifiers. The Gmail spam filter uses it to help protect Gmail inboxes against malicious emails.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">If you would like to use RETVec for your own use cases or research, we created a </span><a href="https://github.com/google-research/retvec/blob/main/notebooks/train_retvec_model_tf.ipynb" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">tutorial</span></a><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to help you get started.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">This research was conducted by Elie Bursztein, Marina Zhang, Owen Vallis, Xinyu Jia, and Alexey Kurakin. We would like to thank Gengxin Miao, Brunno Attorre, Venkat Sreepati, Lidor Avigad, Dan Givol, Rishabh Seth and Melvin Montenegro and all the Googlers who contributed to the project.</span></p><div><span style="font-family: Roboto, sans-serif; font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-28029049329299838882023-11-20T11:49:00.002-05:002023-12-05T05:10:13.670-05:00Two years later: a baseline that drives up security for the industry<span class="byline-author">Royal Hansen, Vice President of Privacy, Safety and Security Engineering, Google</span><div><br /></div><div><span id="docs-internal-guid-b538dc36-7fff-516b-43b3-770257ba33a1"><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Nearly half of third-parties fail to meet two or more of the Minimum Viable Secure Product controls. Why is this a problem? Because "</span><a href="https://resources.securityscorecard.com/research-ungated/close-encounters-of-the-thrird-and-fourth-party-kind" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">98%</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> of organizations have a relationship with at least one third-party that has experienced a breach in the last 2 years."</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In this post, we're excited to share the latest improvements to the Minimum Viable Secure Product (MVSP) controls. We'll also shed light on how adoption of MVSP has helped Google improve its security processes, and hope this example will help motivate third-parties to increase their adoption of MVSP controls and thus improve product security across the industry.</span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 18pt;"><span face="Roboto, sans-serif" style="color: #4285f4; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">About MVSP</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In October 2021, Google </span><a href="https://security.googleblog.com/2021/10/launching-collaborative-minimum.html" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">publicly launched</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> MVSP alongside launch partners. Our original goal remains unchanged: to provide a vendor-neutral application security baseline, designed to eliminate overhead, complexity, and confusion in the end-to-end process of onboarding third-party products and services. It covers themes such as procurement, security assessment, and contract negotiation.</span></p><br /><div style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif2IFt3cieWCzzr-6XtA11xiq158s_htn1XJ6LS9gniO8bSLuoLnHLYiNSbemXDyVZGBKQjiQ2xvHtwpkwIuStMcALW9RNbeff9RMkVrkrsZnBxi4LJqAk6SMx0zgRla44ym5hP8rAXPSjf2lABCvEg9snrLf-yfPSe6jHAVrjARz1IUaBCOQEIYxYFksU/s1227/Explanantion.png"><img alt="What is Minimum Viable Secure Product (MVSP) MVSP is a list of fundamental application security controls that should be integrated into enterprise-ready products and services. The controls are designed to be simple in order to implement and provide a good foundation for building secure and resilient systems and services." border="0" data-original-height="455" data-original-width="1227" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEif2IFt3cieWCzzr-6XtA11xiq158s_htn1XJ6LS9gniO8bSLuoLnHLYiNSbemXDyVZGBKQjiQ2xvHtwpkwIuStMcALW9RNbeff9RMkVrkrsZnBxi4LJqAk6SMx0zgRla44ym5hP8rAXPSjf2lABCvEg9snrLf-yfPSe6jHAVrjARz1IUaBCOQEIYxYFksU/w559-h208/Explanantion.png" width="559" /></a></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="color: #4285f4; font-size: 14pt; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="color: #4285f4; font-size: 14pt; white-space-collapse: preserve;">Improvements since launch</span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As part of MVSP’s annual control review, and our core philosophy of </span><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">evolution over revolution</span><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, the working group sought input from the broader security community to ensure MVSP maintains a balance between security and achievability.</span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As a result of these discussions, we launched </span><a href="https://mvsp.dev/mvsp.en/" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">updated controls</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. Key changes include: expanded guidance around external vulnerability reporting to protect bug hunters, and discouraging additional costs for access to basic security features – inline with CISA’s "</span><a href="https://www.cisa.gov/news-events/news/next-chapter-secure-design" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Secure-by-Design</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">" principles.</span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In 2022, we developed guidance on build process security based on </span><a href="https://slsa.dev" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">SLSA</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, to reflect the importance of supply chain security and integrity.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">From an organizational perspective, in the two years since launching, we've seen the community around MVSP continue to expand. The working group has grown to over 20 global members, helping to diversify voices and broaden expertise. We've also had the opportunity to present and discuss the program with a number of key groups, including an invitation to present at the </span><a href="https://www.unicc.org/what-we-do/common-secure-conference/csc2023/" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">United Nations International Computing Centre – Common Secure Conference</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt; text-align: center;"><span style="border: none; display: inline-block; height: 222px; overflow: hidden; width: 309px;"><img alt="Google at the UNICC conference in Valencia, Spain" height="327" src="https://lh7-us.googleusercontent.com/ZzA3WGQaL1Q5e690Fd81GGFDUK1K6YaY5fkKDDN5Ffo_OD349fZ3t4nSz-2ggBghevQEI13kKLrfgCmNwP3OUkQyvj_xsurzLnNNO_5J-f73gpbkBRPCl155JrL1-RD1jz4R-c611R-WPOMv1q9uadqYD4dl1PZXQGij53NHxt7k_ql4YRikG4uM3k_PE7g=w433-h327" style="margin-left: -99.5085px; margin-top: -47.8053px;" width="433" /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt; text-align: center;"><span style="color: #434343; font-family: "Courier New", monospace; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Google at the UNICC conference in Valencia, Spain</span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 18pt;"><span face="Roboto, sans-serif" style="color: #4285f4; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">How Google uses MVSP</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Since its inception, Google has looked to integrate improvements to our own processes using MVSP as a template. Two years later, we can clearly see the impact through faster procurement processes, streamlined contract negotiations, and improved data-driven decision making.</span></p><h3 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 12pt;"><span face="Roboto, sans-serif" style="color: #4285f4; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Highlights</span></h3><ul style="margin-bottom: 0px; margin-top: 0px; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">After implementing MVSP into key areas of Google's third-party life-cycle, we've observed a 68% reduction in the time required for third-parties to complete assessment process.</span></p></li><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">By embedding MVSP into select procurement processes, Google has increased data-driven decision making in earlier phases of the cycle.</span></p></li><li aria-level="1" dir="ltr" style="color: #434343; font-family: Roboto, sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Aligning our </span><a href="http://business.safety.google/ipa" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">Information Protection Addendum</span></a><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">’s safeguards with MVSP has significantly improved our third-party privacy and security risk management processes.</span></p></li></ul><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt; text-align: center;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border: none; display: inline-block; height: 261px; overflow: hidden; width: 660px;"><img alt="68% time reduction observed after implementing MVSP" height="249" src="https://lh7-us.googleusercontent.com/Tc4V3bsjWFvx170UreVWPpGXMPnVCIUIaKNgScgBcNBODk-OSmhA1W5r8UPvelrIB-ah1YTuBVA8qT1L0Pd3IOxrve5gmB5j4Fjox0KbwP93_hXXsaS_nA7nJ9d990xwiD0j2HeZJA_TAPIZDWAqJjVQUT-DfZTj2EuBNOJ1QmSnCtnnV2kDpRK8QhrIn-E=w630-h249" style="margin-left: 0px; margin-top: 0px;" width="630" /></span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">You use </span><a href="https://mvsp.dev" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">MVSP</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to enhance your software or procurement processes by reviewing some </span><a href="https://mvsp.dev/#use-cases" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">common use-cases</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and adopting them into your third-party risk management and/or contracting workflows .</span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 18pt;"><span face="Roboto, sans-serif" style="color: #4285f4; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">What's next?</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border: none; display: inline-block; height: 329px; overflow: hidden; width: 579px;"><img alt="Security Maturity Levels - Minimum, Basic, Advanced, and Expert" height="320" src="https://lh7-us.googleusercontent.com/cCzC1w5fevEwNnNrdOkG0cdH1mA9wlsKUGUCDzExQw3Kbc1ILtUMIyqny_ZilVucBoh8EAZKF7FgIquAsLYXe51nVfqElxKasWaK_aW2BDZuSEXetMzxz6yqzSABn8tg5kUeAo9Kg43q5BkKEkKZpLNhK26BqVWFQ1Wx3xDo0b-7CJ8HclkIduZcT5GXccU=w562-h320" style="margin-left: 0px; margin-top: 0px;" width="562" /></span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We're invested in helping the industry manage risk posture through continuous improvement, while increasing the minimum bar for product security across the industry.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">By making MVSP available to the wider industry, we are helping to create a solid foundation for growing the maturity level of products and services. Google has benefited from driving security and safety improvements through the use of leveled sets of requirements. We expect the same to be true across the wider industry.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We've seen success, but there is still work to be done. Based on initial observations, as mentioned above, 48% of third-parties fail to meet two or more of the Minimum Viable Secure Product controls.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt; text-align: center;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border: none; display: inline-block; height: 281px; overflow: hidden; width: 523px;"><img alt="48% of third parties fail to meet two or more MVSP controls" height="274" src="https://lh7-us.googleusercontent.com/-mgNRY0pFuYVb4OUfHg8bON7UiyStXntWVr8BJAXvUtHpIcRlW05dzxxeiglOU2Lbyic5dfDflBODILLuXQ1mlCe-zPs4VmppISB51DCHQrujWPntvGOmn6lLKB3S4WOZdIMXfcIQj7CseaGvvLjketyMjPNL1RVpUQrZgAJfTVWg5CvKRKQiRYUkCdEkGs=w513-h274" style="margin-left: 0px; margin-top: 0px;" width="513" /></span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As an industry, we can't stand still when it comes to product security. Help us raise the minimum bar for application security by adopting MVSP and ensuring we as an industry don’t accept anything less than a strong security baseline that works for the wider industry.</span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 18pt;"><span face="Roboto, sans-serif" style="color: #4285f4; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Acknowledgements</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Google and the MVSP working group would like to thank those who have supported and contributed since its inception. If you'd like to get involved or provide feedback, please </span><a href="https://github.com/vendorsec/mvsp" style="text-decoration-line: none;"><span face="Roboto, sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">reach out</span></a><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 10pt; margin-top: 0pt;"><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Thank you to Chris John Riley, Gabor Acs-Kurucz, Michele Chubirka, Anna Hupa, Dirk Göhmann and Kaan Kivilcim from the Google MVSP Group for their contributions to this post.</span></p><div><span face="Roboto, sans-serif" style="color: #434343; font-size: 12pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-24922792007724323332023-11-08T09:00:00.001-05:002023-11-08T09:03:58.284-05:00Evolving the App Defense Alliance<span class="byline-author">Posted by Nataliya Stanetsky, Android Security and Privacy Team</span>
<p>
The App Defense Alliance (ADA), an industry-leading collaboration <a href="https://security.googleblog.com/2019/11/the-app-defense-alliance-bringing.html">launched</a> by Google in 2019 dedicated to ensuring the safety of the app ecosystem, is taking a major step forward. We are proud to <a href="https://www.linuxfoundation.org/press/app-defense-alliance-migrates-under-jdf-with-google-meta-microsoft-as-steering-committee" target="_blank">announce</a> that the App Defense Alliance is moving under the umbrella of the Linux Foundation, with Meta, Microsoft, and Google as founding steering members.
</p>
<p>
This strategic migration represents a pivotal moment in the Alliance’s journey, signifying a shared commitment by the members to strengthen app security and related standards across ecosystems. This evolution of the App Defense Alliance will enable us to foster more collaborative implementation of industry standards for app security.
</p>
<p>
<strong>Uniting for App Security</strong>
</p>
<p>
The digital landscape is continually evolving, and so are the threats to user security. With the ever-increasing complexity of mobile apps and the growing importance of data protection, now is the perfect time for this transition. The Linux Foundation is renowned for its dedication to fostering open-source projects that drive innovation, security, and sustainability. By combining forces with additional members under the Linux Foundation, we can adapt and respond more effectively to emerging challenges.
</p>
<p>
The commitment of the newly structured App Defense Alliance’s founding steering members – Meta, Microsoft, and Google – is pivotal in making this transition a reality. With a member community spanning an additional 16 General and Contributor Members, the Alliance will support industry-wide adoption of app security best practices and guidelines, as well as countermeasures against emerging security risks.
</p>
<p>
<strong>Continuing the Malware Mitigation Program</strong>
</p>
<p>
The App Defense Alliance was formed with the mission of reducing the risk of app-based malware and better protecting Android users. Malware defense remains an important focus for Google and Android, and we will continue to partner closely with the Malware Mitigation Program members – ESET, Lookout, McAfee, Trend Micro, Zimperium – on direct signal sharing. The migration of ADA under the Linux Foundation will enable broader threat intelligence sharing across leading ecosystem partners and researchers.
</p>
<p>
<strong>Looking Ahead and Connecting With the ADA</strong>
</p>
<p>
We invite you to stay <a href="http://appdefensealliance.org">connected with the newly structured App Defense Alliance</a> under the Linux foundation umbrella. Join the conversation to help make apps more secure. Together with the steering committee, alliance partners, and the broader ecosystem, we look forward to building more secure and trustworthy app ecosystems.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-78489023063139891952023-11-07T13:56:00.002-05:002023-11-07T14:06:03.608-05:00MTE - The promising path forward for memory safety<span class="byline-author">Posted by Andy Qin, Irene Ang, Kostya Serebryany, Evgenii Stepanov </span>
<p>
Since 2018, Google has <a href="https://security.googleblog.com/2019/08/adopting-arm-memory-tagging-extension.html">partnered with ARM</a> and collaborated with many ecosystem partners (SoCs vendors, mobile phone OEMs, etc.) to develop Memory Tagging Extension (MTE) technology. We are now happy to share the growing adoption in the ecosystem. MTE is now available on some OEM devices (as noted in a recent <a href="https://googleprojectzero.blogspot.com/2023/11/first-handset-with-mte-on-market.html#:~:text=It's%20been%20a%20long%20wait,allows%20you%20to%20enable%20MTE!">blog post</a> by Project Zero) with Android 14 as a developer option, enabling developers to use MTE to discover memory safety issues in their application easily. <br>
</p>
<p>
The security landscape is changing dynamically, new attacks are becoming more complex and costly to mitigate. It’s becoming increasingly important to detect and prevent security vulnerabilities early in the software development cycle and also have the capability to mitigate the security attacks at the first moment of exploitation in production.<br><br>The biggest contributor to security vulnerabilities are memory safety related defects and Google has invested in a set of technologies to help mitigate memory safety risks. These include but are not limited to:<br>
</p>
<ul>
<li><a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">Shifting to memory safe languages</a> such as Rust as a proactive solution to prevent the new memory safety bugs from being introduced in the first place.
<li>Tools for detecting memory safety defects in the development stages and production environment, such as widely used sanitizer technologies<sup id="fnref1"><a href="#fn1" rel="footnote">1</a></sup> (<a href="https://clang.llvm.org/docs/AddressSanitizer.html">ASAN</a>, <a href="https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html">HWASAN</a>, <a href="https://llvm.org/docs/GwpAsan.html">GWP-ASAN</a>, etc.) as well as fuzzing (with sanitizers enabled).
<li>Foundational technologies like MTE, which many experts believe is <a href="https://googleprojectzero.blogspot.com/2023/08/summary-mte-as-implemented.html">the most promising path forward for improving C/C++ software security</a> and it can be deployed both in development and production at reasonably low cost.
</li>
</ul>
</li>
</ul>
<p>
MTE is a hardware based capability that can detect unknown memory safety vulnerabilities in testing and/or mitigate them in production. It works by tagging the pointers and memory regions and comparing the tags to identify mismatches (<a href="https://www.usenix.org/system/files/login/articles/login_summer19_03_serebryany.pdf">details</a>). In addition to the security benefits, MTE can also help ensure integrity because memory safety bugs remain one of the major contributors to silent data corruption that not only impact customer trust, but also cause lost productivity for software developers.
</p>
<p>
At the moment, MTE is supported on some of the latest chipsets:
</p>
<ul>
<li>Focusing on security for Android devices, the MediaTek <a href="https://www.mediatek.com/products/smartphones-2/mediatek-dimensity-9300">Dimensity 9300</a> integrates support for MTE via ARM's latest v9 architecture (which is what Cortex-X4 and Cortex-A720 processors are based on). This feature can be switched on and off in the bootloader by users and developers instead of having it always on or always off.
<li>Tensor G3 integrates support for MTE only within the developer mode toggle. Feature can be activated by developers.
</li>
</ul>
</li>
</ul>
<p>
For both chipsets, this feature can be switched on and off by developers, making it easier to find memory-related bugs during development and after deployment. MTE can help users stay safe while also improving time to market for OEMs.<br><br>Application developers will be the first to leverage this feature as a way to improve their application security and reliability in the software development lifecycle. MTE can effectively help them to discover hard-to-detect memory safety vulnerabilities (buffer overflows, user-after-free, etc.) with clear & actionable stack trace information in integration testing or pre-production environments. Another benefit of MTE is that the engineering cost of memory-safety testing is drastically reduced because heap bug detection (which is majority of all memory safety bugs) does not require any source or binary changes to leverage MTE, i.e. advanced memory-safety can be achieved with just a simple environment or configuration change.<br><br>We believe that MTE will play a very important role in detecting and preventing memory safety vulnerabilities and provide a promising path towards improving software security.
</p>
<!-- Footnotes themselves at the bottom. -->
<h2>Notes</h2>
<div class="footnotes">
<hr>
<ol><li id="fn1">
<p>
ASAN = Address Sanitizer; HWASAN = HW based ASAN;GWP-ASAN = sampling based ASAN <a href="#fnref1" rev="footnote">↩</a>
</ol></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-61542059162417731802023-11-03T14:38:00.001-04:002023-11-03T16:37:53.014-04:00Qualified certificates with qualified risks<span class="byline-author">Posted by Chrome Security team</span>
<p>
Improving the interoperability of web services is an important and worthy goal. We believe that it should be easier for people to maintain and control their digital identities. And we appreciate that policymakers working on European Union digital certificate legislation, known as eIDAS, are working toward this goal. However, a specific part of the legislation, Article 45, hinders browsers’ ability to enforce certain security requirements on certificates, potentially holding back advances in web security for decades. We and many past and present leaders in the international web community have significant concerns about Article 45's impact on security.
</p>
<p>
We urge lawmakers to heed the <a href="https://eidas-open-letter.org/" target="_blank">calls</a> of scientists and security experts to revise this part of the legislation rather than erode users’ privacy and security on the web.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-43812042677421447612023-11-02T12:00:00.000-04:002023-11-02T12:00:24.061-04:00More ways for users to identify independently security tested apps on Google Play<span class="byline-author">Posted by Nataliya Stanetsky, Android Security and Privacy Team</span>
<p>
Keeping Google Play safe for users and developers remains a top priority for Google. As users increasingly prioritize their digital privacy and security, we continue to invest in our <a href="https://blog.google/products/google-play/data-safety/">Data Safety section</a> and transparency labeling efforts to help users make more informed choices about the apps they use.
</p>
<p>
<a href="https://iotsecurityprivacy.org/research">Research</a> shows that transparent security labeling plays a crucial role in consumer risk perception, building trust, and influencing product purchasing decisions. We believe the same principles apply for labeling and badging in the Google Play store. The transparency of an app’s data security and privacy play a key role in a user’s decision to download, trust, and use an app.
</p>
<p>
<strong>Highlighting Independently Security Tested VPN Apps</strong>
</p>
<p>
Last year, App Defense Alliance (ADA) <a href="https://security.googleblog.com/2022/12/app-defense-alliance-expansion.html">introduced</a> MASA (Mobile App Security Assessment), which allows developers to have their apps independently validated against a <a href="https://owasp.org/www-project-mobile-app-security/">global</a> security standard. This signals to users that an independent third-party has validated that the developers designed their apps to meet these industry mobile security and privacy minimum best practices and the developers are going the extra mile to identify and mitigate vulnerabilities. This, in turn, makes it harder for attackers to reach users' devices and improves app quality across the ecosystem. Upon completion of the successful validation, Google Play gives developers the option to declare an “Independent security review” badge in its Data Safety section, as shown in the image below. While certification to baseline security standards does not imply that a product is free of vulnerabilities, the badge associated with these validated apps helps users see at-a-glance that a developer has prioritized security and privacy practices and committed to user safety.
</p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-bur36840wvlCyW9uYY9bV_at1TiM3Va9mBaxxXPEYXHz4Ckoc1p_mLq9OWJ0aLZbB24ePm8CiTcrfI76ozLTkF7slHAhtk3U7oT8sKxnjM-DOUubll2T_a9p9hyPBNZtRpH9cH3hfqVRdTxMtHzMsgAEsP2uK0AajKoANMdRjZ5WphFV3c3H8dNC5G8i/s1600/independent-review-wrapped.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="1600" data-original-width="1333" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-bur36840wvlCyW9uYY9bV_at1TiM3Va9mBaxxXPEYXHz4Ckoc1p_mLq9OWJ0aLZbB24ePm8CiTcrfI76ozLTkF7slHAhtk3U7oT8sKxnjM-DOUubll2T_a9p9hyPBNZtRpH9cH3hfqVRdTxMtHzMsgAEsP2uK0AajKoANMdRjZ5WphFV3c3H8dNC5G8i/s1600/independent-review-wrapped.png"/></a></div>
<p>
To help give users a simplified view of which apps have undergone an <a href="https://appdefensealliance.dev/masa">independent security validation</a>, we’re introducing a new Google Play store banner for specific app types, starting with VPN apps. We’ve launched this banner beginning with VPN apps due to the sensitive and significant amount of user data these apps handle. When a user searches for VPN apps, they will now see a banner at the top of Google Play that educates them about the “Independent security review” badge in the Data Safety Section. Users also have the ability to “Learn More”, which redirects them to the <a href="https://appdefensealliance.dev/directory?category=vpn">App Validation Directory</a>, a centralized place to view all VPN apps that have been independently security reviewed. Users can also discover additional technical assessment details in the App Validation Directory, helping them to make more informed decisions about what VPN apps to download, use, and trust with their data.
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz8TSWR5-nrJ70yg1TBGGf8oUKlZSGkZ0yOWs3U8EcIsThHh1qGXOXK-L9AaRaY9p6ikw01BByQScARRFNZ2jlFUnYbBvoX8b8wOSoqWlBQpckNMkQC3pRL4X-LiYb21V9gyW24Fyc6YEeKLqRd3m3xwbAS3qETWuTk5yUziuU8-yH1W5RF3eN_vuQFrc0/s976/play-store-vpn-wrapped.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" height="600" data-original-height="976" data-original-width="508" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz8TSWR5-nrJ70yg1TBGGf8oUKlZSGkZ0yOWs3U8EcIsThHh1qGXOXK-L9AaRaY9p6ikw01BByQScARRFNZ2jlFUnYbBvoX8b8wOSoqWlBQpckNMkQC3pRL4X-LiYb21V9gyW24Fyc6YEeKLqRd3m3xwbAS3qETWuTk5yUziuU8-yH1W5RF3eN_vuQFrc0/s600/play-store-vpn-wrapped.png"/></a></div>
VPN providers such as NordVPN, Google One, ExpressVPN, and others have already undergone independent security testing and publicly declared the badge showing their good standing with the MASA program. We encourage and anticipate additional VPN app developers to undergo independent security testing, bringing even more transparency to users. If you are a VPN developer and interested in learning more about this feature, please <a href="https://services.google.com/fb/forms/mindependentsecurityreview/">submit this form</a>.
</p>
<p>
<strong>Our Commitment to App Security and Privacy Transparency on Google Play</strong>
</p>
<p>
By encouraging independent security testing and displaying security badges for validated apps, we highlight developers who prioritize user safety and data transparency. We also provide developers with app security and privacy best practices – through <a href="https://www.youtube.com/watch?si=iuIe1B2CKCxFBhCY&v=FuYjzf8lymw&feature=youtu.be">Play PolicyBytes videos</a>, webinars, <a href="https://android-developers.googleblog.com/?sjid=12212977848216103339-NA">blog posts</a>, the <a href="https://support.google.com/googleplay/android-developer/community?hl=en">Developer Help Community</a>, and other resources – in accordance with our developer policies that help keep Google Play safe. We are continually working on improvements to our app review process, policies, and programs to keep users safe and to help developers navigate our policies with ease. To learn more about how we give developers the tools to succeed while keeping users safe on Google Play, read the <a href="https://safety.google/intl/en_us/stories/google-play-safety/">Google Safety Center</a> article.
</p>
<p>
Our efforts to prioritize security and privacy transparency on Google Play are aligned with the needs and expectations we’ve heard from both users and developers. We believe that by prioritizing initiatives that bolster user safety and trust, we can foster a thriving app ecosystem where users can make more informed app decisions and developers are encouraged to uphold the highest standards of security and privacy.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-4194718833642838562023-10-26T08:01:00.008-04:002023-10-26T09:05:09.330-04:00Increasing transparency in AI security<span class="byline-author">Mihai Maruseac, Sarah Meiklejohn, Mark Lodato, Google Open Source Security Team (GOSST)<br /></span><div><br /></div><div><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">New AI innovations and applications are reaching consumers and businesses on an almost-daily basis. Building AI securely is a paramount concern, and we believe that Google’s </span><a href="https://blog.google/technology/safety-security/introducing-googles-secure-ai-framework/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Secure AI Framework</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> (SAIF) can help chart a path for creating AI applications that users can trust. Today, we’re highlighting two new ways to make information about AI supply chain security universally discoverable and verifiable, so that AI can be created and used responsibly. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The first principle of SAIF is to ensure that the AI ecosystem has strong security foundations. In particular, the software supply chains for components specific to AI development, such as machine learning models, need to be secured against threats including </span><a href="https://www.wiz.io/blog/38-terabytes-of-private-data-accidentally-exposed-by-microsoft-ai-researchers" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">model tampering</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, </span><a href="https://www.cobalt.io/blog/data-poisoning-attacks-a-new-attack-vector-within-ai" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">data poisoning</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, and the </span><a href="https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">production of harmful content</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Even as machine learning and artificial intelligence continue to evolve rapidly, some solutions are now within reach of ML creators. We’re building on our prior work with the </span><a href="http://openssf.org" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Open Source Security Foundation</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to show how </span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">ML model creators can and should protect against ML supply chain attacks by using </span><a href="http://slsa.dev" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">SLSA</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"> and </span><a href="http://sigstore.dev" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Sigstore</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Supply chain security for ML</span></h1><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">For supply chain security of conventional software (software that does not use ML), we usually consider questions like:</span></p><br /><p dir="ltr" role="presentation" style="font-family: Arial, sans-serif; font-size: 11pt; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; white-space: pre;"></p><ul style="text-align: left;"><li><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Who published the </span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">software</span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">? Are they trustworthy? Did they use safe practices?</span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">For open source </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">software</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, what was the </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">source</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> code?</span></span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">What </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">dependencies</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> went into </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">building</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">software</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">?</span></span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Could the </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">software</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> have been replaced by a tampered version following publication? Could this have occurred during </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">build</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> time?</span></span></li></ul><p></p><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">All of these questions also apply to the </span><a href="https://www.kaggle.com/models" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">hundreds of free ML models</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that are available for use on the internet. Using an ML model means trusting every part of it, just as you would any other piece of software. This includes concerns such as:</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" role="presentation" style="font-family: Arial, sans-serif; font-size: 11pt; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; white-space: pre;"></p><ul style="text-align: left;"><li><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Who published the </span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">model</span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">? Are they trustworthy? Did they use safe practices?</span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">For open source </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">models</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, what was the </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">training</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> code?</span></span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">What </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">datasets</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> went into </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">training</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">model</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">?</span></span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Could the </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">model</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> have been replaced by a tampered version following publication? Could this have occurred during </span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">training</span><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> time?</span></span></li></ul><p></p><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">We should treat tampering of ML models with the same severity as we treat injection of malware into conventional software.</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> In fact, since </span><a href="https://github.com/tensorflow/tensorflow/blob/master/SECURITY.md#tensorflow-models-are-programs" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">models are programs</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, many allow the same types of arbitrary code execution exploits that are leveraged for attacks on conventional software. Furthermore, a tampered model could leak or steal data, cause harm from biases, or spread dangerous misinformation. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Inspection of an ML model is insufficient to determine whether bad behaviors were injected. This is similar to trying to reverse engineer an executable to identify malware. To protect supply chains at scale, we need to know </span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">how</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> the model or software was created to answer the questions above.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Solutions for ML supply chain security</span></h1><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In recent years, we’ve seen how providing </span><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">public</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and </span><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">verifiable</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> information about what happens during different stages of software development is an effective method of protecting conventional software against supply chain attacks. This supply chain transparency offers protection and insights with:</span></p><div><span><br /></span></div><br /><p dir="ltr" role="presentation" style="font-family: Arial, sans-serif; font-size: 11pt; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; white-space: pre;"></p><ul style="text-align: left;"><li><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Digital signatures, such as those from </span><a href="http://sigstore.dev" style="text-decoration-line: none;"><span style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">Sigstore</span></a><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">, which allow users to verify that the software wasn’t tampered with or replaced</span></li><li><span id="docs-internal-guid-f5bde882-7fff-1291-a841-a57ca65fb1f9"><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Metadata such as </span><a href="https://slsa.dev/provenance/v1" style="font-family: Arial, sans-serif; font-size: 11pt; text-decoration-line: none; white-space: pre;"><span style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">SLSA provenance</span></a><span style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that tell us what’s in software and how it was built, allowing consumers to ensure license compatibility, identify known vulnerabilities, and detect more advanced threats</span></span></li></ul><p></p><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Together, these solutions help combat </span><a href="https://www.sonatype.com/state-of-the-software-supply-chain/introduction?utm_campaign=pr&utm_source=pressrelease&utm_medium=organic" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">the enormous uptick in supply chain attacks</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that have turned every step in the software development lifecycle into a potential target for malicious activity.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We believe transparency throughout the development lifecycle will also help secure ML models, since ML model development </span><a href="https://www.mandiant.com/resources/blog/securing-ai-pipeline#:~:text=pipeline%20we%20used%3A-,AI%20Pipeline,-As%20you%20can" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">follows a similar lifecycle</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> as for regular software artifacts:</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border: none; display: inline-block; height: 343px; overflow: hidden; width: 624px;"><img height="343" src="https://lh7-us.googleusercontent.com/Tg6rzsbRdX03MwJLA3qurE1WQgHU7zqYLhbdKE8ZSX0uy8nveHUGbRaXsLgWGIj6ubm33-TYh-r2k0FLK3LOMt7y8sPPBFxApuMWLaPv24NIKfkEpwrdfXPu6Mmub8dYFSWQ3M4eygNHpNEzMONMy28t5hg0VrEohXOBvIr9-zS2A_WCZmpF0hHg9XBek24" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Similarities between software development and ML model development</span></p><div><span><br /></span></div><div><span><br /></span></div><br /><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">An ML training process can be thought of as a “build:” it transforms some input data to some output data. Similarly, training data can be thought of as a “dependency:” it is data that is used during the build process. Because of the similarity in the development lifecycles, the same software supply chain attack vectors that threaten software development also apply to model development: </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="border: none; display: inline-block; height: 252px; overflow: hidden; width: 624px;"><img height="252" src="https://lh7-us.googleusercontent.com/VGJOwLwJlxm-yuoaokDAFELH6c9X1v8bfhPcHrg5mGibpt-A88G8xZlpBQe13tKdT_V3oQSW3Pkgj19Vs5alQs7eevXVBb4LBRtMtaBz55k0qGnv7SO1YewzaJquU3Cat1CrGiVP3BeTwP8WDLeUt6tApsgl59C4rPnoqxzaa8RAILF5uufLR6mDWM4ewGc" style="margin-left: 0px; margin-top: 0px;" width="624" /></span></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Attack vectors on ML through the lens of the ML supply chain</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: center;"><span face="Arial, sans-serif" style="font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Based on the similarities in development lifecycle and threat vectors, </span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">we propose applying the same supply chain solutions from SLSA and Sigstore to ML models to similarly protect them against supply chain attacks.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Sigstore for ML models</span></h1><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Code signing is a critical step in supply chain security. It identifies the producer of a piece of software and prevents tampering after publication. But normally code signing is difficult to set up—producers need to manage and rotate keys, set up infrastructure for verification, and instruct consumers on how to verify. Often times secrets are also leaked since security is hard to get right during the process.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We suggest bypassing these challenges by using </span><a href="https://www.sigstore.dev/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Sigstore</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, a collection of tools and services that make code signing secure and easy. Sigstore allows any software producer to sign their software by simply using an OpenID Connect token bound to either a workload or developer identity—all without the need to manage or rotate long-lived secrets.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">So how would signing ML models benefit users? By signing models after training, we can assure users that they have the exact model that the builder (aka “trainer”) uploaded. Signing models discourages </span><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">model hub owners from swapping models, addresses the issue of a model hub compromise, an</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">d can help prevent users from being tricked into using a bad model. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: left;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Model signatures make attacks similar to </span><a href="https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">PoisonGPT</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> detectable. The tampered models will either fail signature verification or can be directly traced back to the malicious actor. Our current work to encourage this industry standard includes:</span></p><div style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; text-align: left;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div></span><p style="text-align: left;"><span><br /></span></p><span><p role="presentation" style="font-family: Arial, sans-serif; font-size: 11pt; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; white-space: pre;"></p><ul style="text-align: left;"><li><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Having ML frameworks integrate signing and verification in the model save/load APIs</span></li><li><span><span face="Arial, sans-serif" style="font-size: 11pt; white-space-collapse: preserve;">Having ML model hubs add a badge to all signed models, thus guiding users towards signed models and incentivizing signatures from model developers</span></span></li><li><span><span face="Arial, sans-serif" style="font-size: 11pt; white-space-collapse: preserve;">Scaling model signing for LLMs </span></span></li></ul><p></p><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><div><span face="Arial, sans-serif"><span style="font-size: 14.6667px; white-space-collapse: preserve;"><br /></span></span></div><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">SLSA for ML Supply Chain Integrity</span></h1><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Signing with Sigstore provides users with confidence in the models that they are using, but it cannot answer every question they have about the model. SLSA goes a step further to provide more meaning behind those signatures. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><a href="https://slsa.dev/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">SLSA</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> (Supply-chain Levels for Software Artifacts) is a specification for describing how a software artifact was built. SLSA-enabled build platforms implement controls to prevent tampering and output signed </span><a href="https://slsa.dev/spec/v1.0/provenance" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">provenance</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> describing how the software artifact was produced, including all build inputs. This way, SLSA provides trustworthy metadata about what went into a software artifact.</span></p><div><span><br /></span></div><div><span><br /></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Applying SLSA to ML could provide similar information about an ML model’s supply chain and address attack vectors not covered by model signing, such as compromised source control, compromised training process, and vulnerability injection.</span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"> </span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our vision is to include specific ML information in a SLSA provenance file, which would help users spot an undertrained model or one trained on bad data. Upon detecting a vulnerability in an ML framework, users can quickly identify which models need to be retrained, thus reducing costs.</span></p><div><span><br /></span></div><div><span><br /></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We don’t need special ML extensions for SLSA. Since an ML training process is a build (shown in the earlier diagram), we can apply the existing SLSA guidelines to ML training. The ML training process should be hardened against tampering and output provenance just like a conventional build process. </span><a href="https://github.com/slsa-framework/slsa/issues/978" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">More work</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> on SLSA is needed to make it fully useful and applicable to ML, particularly around describing dependencies such as datasets and pretrained models. Most of these efforts will also benefit conventional software.</span></p><div><span><br /></span></div><div><span><br /></span></div><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">For models training on pipelines that do not require GPUs/TPUs, using an existing, SLSA-enabled build platform is a simple solution. For example, </span><a href="https://cloud.google.com/software-supply-chain-security/docs/safeguard-builds" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Google Cloud Build</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, </span><a href="https://github.com/slsa-framework/slsa-github-generator" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">GitHub Actions</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, or </span><a href="https://about.gitlab.com/blog/2022/11/30/achieve-slsa-level-2-compliance-with-gitlab/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">GitLab CI</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> are all generally available SLSA-enabled build platforms. It is possible to run an ML training step on one of these platforms to make all of the built-in supply chain security features available to conventional software.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h1 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 20pt;"><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">How to do model signing and SLSA for ML today</span></h1><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><span face="Arial, sans-serif" style="font-size: 20pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">By incorporating supply chain security into the ML development lifecycle now, while the problem space is still unfolding, we can jumpstart work with the open source community to establish industry standards to solve pressing problems. This effort is already underway and available for testing. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our repository of tooling for model signing and experimental SLSA provenance support for smaller ML models is </span><a href="https://github.com/google/model-transparency" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">available now</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. Our future ML framework and model hub integrations will be released in this repository as well. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We welcome collaboration with the ML community and are looking forward to reaching consensus on how to best integrate supply chain protection standards into existing tooling (such as </span><a href="https://modelcards.withgoogle.com/about" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Model Cards</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">). If you have feedback or ideas, please feel free to </span><a href="https://github.com/google/model-transparency/issues/new" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">open an issue</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and let us know. </span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-77216510505849272412023-10-26T08:00:00.001-04:002023-10-26T08:00:33.010-04:00Google’s reward criteria for reporting bugs in AI products <span class="byline-author">Eduardo Vela, Jan Keller and Ryan Rinaldi, Google Engineering </span><div><br /></div><div><span id="docs-internal-guid-6fab1431-7fff-735f-bbc1-c09ebb5d9280"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In September, we </span><a href="https://static.googleusercontent.com/media/publicpolicy.google/en//resources/whcommitments.pdf" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">shared</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> how we are implementing the </span><a href="https://www.whitehouse.gov/wp-content/uploads/2023/07/Ensuring-Safe-Secure-and-Trustworthy-AI.pdf" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="background-color: white; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">voluntary AI commitments</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> that we and others in industry made at the White House in July. One of the most important developments involves expanding our existing </span><a href="http://bughunters.google.com/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Bug Hunter Program</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to foster third-party discovery and reporting of issues and vulnerabilities specific to our AI systems. Today, we’re publishing more details on these new reward program elements for the first time. Last year we issued over </span><a href="https://security.googleblog.com/2023/02/vulnerability-reward-program-2022-year.html" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">$12 million in rewards to security researchers</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> who tested our products for vulnerabilities, and we expect today’s announcement to fuel even greater collaboration for years to come. </span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;">What’s in scope for rewards</span><span face="Arial, sans-serif" style="background-color: white; font-size: 14pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline; white-space-collapse: preserve;"> </span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In our recent </span><a href="https://services.google.com/fh/files/blogs/google_ai_red_team_digital_final.pdf" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="background-color: white; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">AI Red Team report</span></a><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, we identified common tactics, techniques, and procedures (TTPs) that we consider most relevant and realistic for </span><a href="https://www.mandiant.com/resources/blog/threat-actors-generative-ai-limited" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="background-color: white; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">real-world adversaries to use against AI systems</span></a><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. The following table incorporates shared learnings from </span><a href="https://blog.google/technology/safety-security/googles-ai-red-team-the-ethical-hackers-making-ai-safer/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="background-color: white; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Google’s AI Red Team</span></a><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> exercises to help the research community better understand what’s in scope for our reward program. We're detailing our criteria for AI bug reports to assist our bug hunting community in effectively testing the safety and security of AI products. Our scope aims to facilitate testing for traditional security vulnerabilities as well as risks specific to AI systems. It is important to note that reward amounts are dependent on severity of the attack scenario and the type of target affected (go </span><a href="https://bughunters.google.com/about/rules/6625378258649088/google-and-alphabet-vulnerability-reward-program-vrp-rules" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="background-color: white; color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">here</span></a><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> for more information on our reward table). </span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><div><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><span id="docs-internal-guid-b8fbe082-7fff-ff13-1ba5-e1d8efcc4b00"><div align="left" dir="ltr" style="margin-left: 9.75pt;"><table style="border-collapse: collapse; border: none;"><colgroup><col width="168"></col><col width="405"></col><col width="133"></col></colgroup><tbody><tr style="height: 0pt;"><td style="background-color: #d9d9d9; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Category</span></p></td><td style="background-color: #d9d9d9; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Attack Scenario</span></p></td><td style="background-color: #d9d9d9; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Guidance</span></p></td></tr><tr style="height: 21pt;"><td rowspan="4" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>Prompt Attacks: </b>Crafting adversarial prompts that allow an adversary to influence the behavior of the model, and hence the output in ways that were not intended by the application.</span></p></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Prompt injections that are invisible to victims and change the state of the victim's account or or any of their assets.</span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Prompt injections into any tools in which the response is used to make decisions that directly affect victim users.</span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Prompt or preamble extraction in which a user is able to extract the initial prompt used to prime the model only when sensitive information is present in the extracted preamble.</span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Using a product to generate violative, misleading, or factually incorrect content in your own session: e.g. 'jailbreaks'. This includes 'hallucinations' and factually inaccurate responses. Google's generative AI products already have a dedicated reporting channel for these types of content issues.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td rowspan="2" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>Training Data Extraction: </b>Attacks that are able to successfully reconstruct verbatim training examples that contain sensitive information. Also called membership inference.</span></p><br /></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Training data extraction that reconstructs items used in the training data set that leak sensitive, non-public information.</span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Extraction that reconstructs nonsensitive/public information.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td rowspan="2" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>Manipulating Models: </b>An attacker able to covertly change the behavior of a model such that they can trigger pre-defined adversarial behaviors.</span></p><br /></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Adversarial output or behavior that an attacker can reliably trigger via specific input in a model owned and operated by Google ("backdoors"). Only in-scope when a model's output is used to change the state of a victim's account or data. </span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Attacks in which an attacker manipulates the training data of the model to influence the model’s output in a victim's session according to the attacker’s preference. Only in-scope when a model's output is used to change the state of a victim's account or data. </span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td rowspan="2" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>Adversarial Perturbation:</b> Inputs that are provided to a model that results in a deterministic, but highly unexpected output from the model.</span></p></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Contexts in which an adversary can reliably trigger a misclassification in a security control that can be abused for malicious use or adversarial gain. </span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Contexts in which a model's incorrect output or classification does not pose a compelling attack scenario or feasible path to Google or user harm.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td rowspan="2" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>Model Theft / Exfiltration: </b>AI models often include sensitive intellectual property, so we place a high priority on protecting these assets. Exfiltration attacks allow attackers to steal details about a model such as its architecture or weights.</span></p></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Attacks in which the exact architecture or weights of a confidential/proprietary model are extracted.</span></p></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Attacks in which the architecture and weights are not extracted precisely, or when they're extracted from a non-confidential model.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td rowspan="4" style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">If you find a flaw in an AI-powered tool other than what is listed above, you can still submit, provided that it meets the </span><a href="https://bughunters.google.com/about/rules/6625378258649088/google-and-alphabet-vulnerability-reward-program-vrp-rules" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">qualifications listed on our program page</span></a><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">.</span></p><br /></td><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">A bug or behavior that clearly meets our </span><a href="https://bughunters.google.com/about/rules/6625378258649088/google-and-alphabet-vulnerability-reward-program-vrp-rules" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">qualifications</span></a><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> for a valid security or abuse issue.</span></p><br /></td><td style="background-color: #d9ead3; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Google Sans, sans-serif"><span style="font-size: 13.3333px; white-space-collapse: preserve;">Using an AI product to do something potentially harmful that is already possible with other tools. For example, finding a vulnerability in open source software (already possible using publicly-available <a href="https://en.wikipedia.org/wiki/Static_application_security_testing">static analysis tools</a>) and producing the answer to a harmful question when the answer is already available online.</span></span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As consistent with our program, issues that we already know about are not eligible for reward.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr><tr style="height: 21pt;"><td style="border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Potential copyright issues: findings in which products return content appearing to be copyright-protected. Google's generative AI products already have a dedicated reporting channel for these types of content issues.</span></p></td><td style="background-color: #f4cccc; border-bottom: solid #000000 1pt; border-color: rgb(0, 0, 0); border-left: solid #000000 1pt; border-right: solid #000000 1pt; border-style: solid; border-top: solid #000000 1pt; border-width: 1pt; overflow-wrap: break-word; overflow: hidden; padding: 5pt; vertical-align: top;"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="background-color: transparent; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Out of Scope</span></p></td></tr></tbody></table></div></span></div><div><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><div><br /></div><div><br /></div><div><br /></div><div><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="font-size: 14pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline;">Conclusion</span><span style="font-size: 14pt; font-style: italic; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; vertical-align: baseline;"> </span></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span id="docs-internal-guid-cfbc9148-7fff-6c4f-b984-45e33889e4ed"></span></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="background-color: white; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We look forward to continuing our work with the research community to discover and fix security and abuse issues in our </span><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">AI-powered features. If you find a qualifying issue, please go to our </span><a href="https://bughunters.google.com/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Bug Hunter website</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> to send us your bug report and–if the issue is found to be valid–be rewarded for helping us keep our users safe.</span></p></div></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-57633425202070476442023-10-25T08:00:00.004-04:002023-10-27T11:41:24.004-04:00Joint Industry statement of support for Consumer IoT Security Principles<span class="byline-author">David Kleidermacher, VP Engineering, Android Security & Privacy and DSPA Security & Privacy, and Eugene Liderman, Director, Android Security Strategy</span><div><br /></div><div><br /></div><div><span id="docs-internal-guid-9716af77-7fff-4dc4-c361-31da3383896d"><p dir="ltr" style="line-height: 1.2; margin-bottom: 13pt; margin-top: 13pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Last week at </span><a href="https://www.sicw.gov.sg/" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Singapore International Cyber Week</span></a><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and the </span><a href="https://www.etsi.org/events/2155-etsi-security-conference-2023" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">ETSI Security Conference</span></a><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">s, the international community gathered together to discuss cybersecurity hot topics of the day. Amidst a number of important cybersecurity discussions, we want to highlight progress on connected device security demonstrated by joint <a href="https://services.google.com/fh/files/misc/iot_principles_for_consumer_iot_securitytransparency.pdf">industry principles for IoT security transparency.</a> The future of connected devices offers tremendous potential for innovation and quality of life improvements. Putting a spotlight on consumer IoT security is a key aspect of achieving these benefits. Marketplace competition can be an important driver of security improvements, with consumers empowered and motivated to make informed purchasing decisions based on device security. </span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 13pt; margin-top: 13pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As with other IoT security transparency initiatives globally, it’s great to see this topic being covered at both conferences this week. The below IoT security labeling principles are aimed at helping to improve consumer awareness and to foster marketplace competition based on security.</span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 13pt; margin-top: 13pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">To help consumers make an informed purchase decision they should receive clear, consistent, and actionable information about the security of the device (e.g. security support period, authentication support, cryptographic assurance) before purchase - a communication and transparency mechanism commonly referred to as “a label” or “labeling,” although the communication is not merely a printed sticker on physical product packaging. While an IoT label will not solve the problem of IoT security on its own, transparency can both help educate consumers and also facilitate the coordination of security responsibilities between all of the components in a connected device ecosystem.</span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 13pt; margin-top: 13pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Our goal is to strengthen the security of IoT devices and ecosystems to protect individuals and organizations, and to unleash the full future benefit of IoT. Security labeling programs can support consumer purchase decisions that drive security improvements, but only if the label is credible, actionable, and easily understood. We are hopeful that the public sector and industry can work together to drive harmonized policies that achieve this goal. </span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 27pt; margin-right: 36pt; margin-top: 24pt;"><span face=""Google Sans", sans-serif" style="font-size: 10pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Signed,</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Google</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">ARM</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Assa Abloy</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Finite State</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">HackerOne</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Keysight</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">NXP</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">OpenPolicy</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Rapid7</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; color: black; font-family: Arial,sans-serif; font-size: 10pt; font-style: italic; font-variant: normal; font-weight: 400; text-decoration: none; vertical-align: baseline; white-space: pre-wrap; white-space: pre;">Schlage</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: Arial, sans-serif; font-size: 10pt; font-style: italic; white-space-collapse: preserve;">Silicon Labs</span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-4462066957071090642023-10-18T12:00:00.000-04:002023-10-18T12:00:27.061-04:00Enhanced Google Play Protect real-time scanning for app installs <span class="byline-author">Posted by Steve Kafka, Group Product Manager and Roman Kirillov, Senior Engineering Manager</span>
<p>
Mobile devices have supercharged our modern lives, helping us do everything from purchasing goods in store and paying bills online to storing financial data, health records, passwords and pictures. According to Data.ai, the pandemic <a href="https://www.data.ai/en/insights/market-data/state-of-mobile-2022-bouyant-economy/">accelerated</a> existing mobile habits – with app categories like finance growing 25% year-over-year and users spending over 100 billion hours in shopping apps. It's now even more important that data is protected so that bad actors can't access the information.
</p>
<p>
<strong>Powering up Google Play Protect</strong><br>
</p>
<p>
<a href="https://support.google.com/googleplay/answer/2812853?hl=en">Google Play Protect</a> is built-in, proactive protection against malware and unwanted software and is enabled on all Android devices with Google Play Services. Google Play Protect scans 125 billion apps daily to help protect you from malware and unwanted software. If it finds a potentially harmful app, Google Play Protect can take certain actions such as sending you a warning, preventing an app install, or disabling the app automatically.
</p>
<p>
To try and avoid detection by services like Play Protect, cybercriminals are using novel malicious apps available outside of Google Play to infect more devices with polymorphic malware, which can change its identifiable features. They’re turning to social engineering to trick users into doing something dangerous, such as revealing confidential information or downloading a malicious app from ephemeral sources – most commonly via links to download malicious apps or downloads directly through messaging apps.
</p>
<p>
For this reason, Google Play Protect has always also offered users protection outside of Google Play. It checks your device for potentially harmful apps regardless of the install source when you’re online or offline as well. Previously, when installing an app, Play Protect conducted a real-time check and warned users when it identified an app known to be malicious from existing scanning intelligence or was identified as suspicious from our on-device machine learning, similarity comparisons, and other techniques that we are always evolving.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-d8w4AWUU2dir9a0LiPLPaQqMSy07is-WdGiuBxx2EfI7ooJa19Iq6HQ5gd3fFupghHfZIHkdpVvS73G4-4WW808TkaoNGIjwNFZQreSoLfQ9H603SLwmwX1sy6feNckC-AqTPRSXlRntBd43bMK9MZUuRAQVSPdOw8-wpNqB_zHD4xVRhD_joWMhhm_F/s1600/GPP-real-time-scanning-3.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="2826" data-original-width="4320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-d8w4AWUU2dir9a0LiPLPaQqMSy07is-WdGiuBxx2EfI7ooJa19Iq6HQ5gd3fFupghHfZIHkdpVvS73G4-4WW808TkaoNGIjwNFZQreSoLfQ9H603SLwmwX1sy6feNckC-AqTPRSXlRntBd43bMK9MZUuRAQVSPdOw8-wpNqB_zHD4xVRhD_joWMhhm_F/s1600/GPP-real-time-scanning-3.png"/></a></div>
</p>
<p>
<br>
</p>
<p>
Today, we are making Google Play Protect’s security capabilities even more powerful with real-time scanning at the code-level to combat novel malicious apps. Google Play Protect will now recommend a real-time app scan when installing apps that have never been scanned before to help detect emerging threats.
</p>
<p>
Scanning will extract important signals from the app and send them to the Play Protect backend infrastructure for a code-level evaluation. Once the real-time analysis is complete, users will get a result letting them know if the app looks safe to install or if the scan determined the app is potentially harmful. This enhancement will help better protect users against malicious polymorphic apps that leverage various methods, such as AI, to be altered to avoid detection.
</p>
<p>
Our security protections and machine learning algorithms learn from each app <a href="https://developers.google.com/android/play-protect/cloud-based-protections">submitted to Google for review</a> and we look at thousands of signals and compare app behavior. Google Play Protect is constantly improving with each identified app, allowing us to strengthen our protections for the entire Android ecosystem.
</p>
<p>
This enhancement to Google Play Protect has started to roll out to all Android devices with Google Play services in select countries, starting with India, and will expand to all regions in the coming months.
</p>
<p>
<strong>Our Multi-Layered User Protections on Android</strong>
</p>
<p>
<br>Android takes a multi-layered defense approach to help keep you safe from mobile malware and unwanted software on Android. Android’s built-in <a href="https://www.android.com/safety/security/">proactive and advanced user protections</a> like Google Play Protect, <a href="https://support.google.com/android/answer/7680439?hl=en">ongoing security updates</a>, <a href="https://support.google.com/android/answer/9431959?hl=en">app permission controls</a>, <a href="https://safebrowsing.google.com/">Safe Browsing</a>, and more – alongside spam and phishing protection in Messages by Google and Gmail – work together to help protect your data security and privacy. We are constantly improving this multi-layered approach to find new ways to protect our billions of users.
</p>
<p>
Keeping Android users safe is a top priority. We are committed to working with our ecosystem partners and app developer community to improve the security of apps and combat malware and unwanted software to make Android even more secure.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-41837840529010705902023-10-10T15:39:00.006-04:002023-10-17T15:11:10.823-04:00Scaling BeyondCorp with AI-Assisted Access Control Policies<span class="byline-author">Ayush Khandelwal, Software Engineer, Michael Torres, Security Engineer, Hemil Patel, Technical Program Manager, Sameer Ladiwala, Software Enginner</span><div><br /></div><div><br /></div><div><span id="docs-internal-guid-a48468cd-7fff-9843-1d9e-9511924ea894"><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In July 2023, four Googlers from the Enterprise Security and Access Security organizations developed a tool that aimed at revolutionizing the way Googlers interact with Access Control Lists - SpeakACL. This tool, awarded the Gold Prize during Google’s internal Security & AI Hackathon, allows developers to create or modify security policies using simple English instructions rather than having to learn system-specific syntax or complex security principles. This can save security and product teams hours of time and effort, while helping to protect the information of their users by encouraging the reduction of permitted access by adhering to the principle of least privilege.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;"><br /></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span face="Arial, sans-serif" style="font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Access Control Policies in BeyondCorp</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Google requires developers and owners of enterprise applications to define their own access control policies, as described in </span><a href="https://research.google/pubs/pub45728/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">BeyondCorp: The Access Proxy</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. We have invested in reducing the difficulty of self-service ACL and ACL test creation to encourage these service owners to define least privilege access control policies. However, it is still challenging to concisely transform their intent into the language acceptable to the access control engine. Additional complexity is added by the variety of engines, and corresponding policy definition languages that target different access control domains (i.e. websites, networks, RPC servers).</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">To adequately implement an access control policy, service developers are expected to learn various policy definition languages and their associated syntax, in addition to sufficiently understanding security concepts. As this takes time away from core developer work, it is not the most efficient use of developer time. A solution was required to remove these challenges so developers can focus on building innovative tools and products.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span face="Arial, sans-serif" style="font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Making it Work</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We built a prototype interface for interactively defining and modifying access control policies for the </span><a href="https://research.google/pubs/pub45728/" style="text-decoration-line: none;"><span face="Arial, sans-serif" style="color: #1155cc; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">BeyondCorp access control engine</span></a><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> using the PaLM 2 Large Language Model (LLM). using the PaLM 2 Large Language Model (LLM). We used Google Colab to provide the model with a diverse, highly variable, dataset using in-context learning and fine-tuning. In-context learning allows the model to learn from a dataset of examples that are relevant to the task at hand, which we provided via few-shot learning. Fine-tuning allows the model to be adapted to a specific task by adjusting its parameters. Tuning the model with a diverse labeled dataset that we curated for this task allowed us to improve its ability to generate ACLs that are both syntactically accurate and adhered to the principle of least privilege. </span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">With SpeakACL, and other tools leveraging AI in security, it is always recommended to take a conservative approach with the autonomy you give an AI agent. To ensure our model outputs are correct & safe to use, we combined our tool with existing safeguards that exist at Google for all access policy modifications:</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><ul style="margin-bottom: 0px; margin-top: 0px; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Request LGTM from a teammate</span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-wrap: wrap; vertical-align: baseline;"> to ensure that the intent of the proposed change is correct. </span></p></li></ul><ul style="margin-bottom: 0px; margin-top: 0px; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">Automated Risk Assessment </span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">occurs on proposed security policy at Google. </span></p></li><li aria-level="1" dir="ltr" style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Manual Review by Security Engineers</span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; text-wrap: wrap; vertical-align: baseline;"> is performed on changes not assessed as low risk to ensure compliance with security policies and guidelines.</span></p></li><li aria-level="1" dir="ltr" style="font-family: Arial, sans-serif; font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 700; text-wrap: wrap; vertical-align: baseline;">Linting, unit tests, and integration tests</span><span style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> ensure that the access control language syntax is correct, and that the change does not break any expected access or permit unexpected access.</span></p></li></ul><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span face="Arial, sans-serif" style="font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></h2><div><span face="Arial, sans-serif" style="font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div><h2 dir="ltr" style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;"><span face="Arial, sans-serif" style="font-size: 16pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; font-weight: 400; vertical-align: baseline; white-space-collapse: preserve;">Looking to the future</span></h2><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">While progress in AI is impressive, it is crucial we as an industry continue to prioritize safety while navigating the landscape. Other than adding checks to syntactically and semantically verify access policies produced by our model, we also designed safeguards for sensitive information disclosure, data leaking, prompt injections, and supply chain vulnerabilities to make sure our model is performing at the highest level of security.</span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">SpeakACL is an ACL Generation tool that has the potential to revolutionize the way access policies are created and managed. The efficiency, security, and ease of use achieved by this AI-powered ACL Generation Engine reflects Google’s ongoing commitment to leveraging AI across domains to develop cutting-edge products and infrastructure. </span></p><div><span face="Arial, sans-serif" style="font-size: 11pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></div></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-44597089575245036072023-10-09T12:30:00.002-04:002023-10-10T10:36:43.939-04:00Bare-metal Rust in Android<span class="byline-author">Posted by Andrew Walbran, Android Rust Team</span>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOWhVNlsU5h9gIRvSXayexpRnohm0jHgR6qi7Yp9PZGavrZR1-wtaCKvz2AWnnjjoSH19rhYG_E4L4MeUtlIm9kD3iqeisl6XCpwVyzGt3f6l-heFMaVNtOBFVRN7a2eykmTk-KGyR6Dt7VnHJsLD2poI3350g5o1bpbRdtdfpqIop55Uv8liiTN5lL7Bv/s1600/image1%20%284%29.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="620" data-original-width="1162" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhOWhVNlsU5h9gIRvSXayexpRnohm0jHgR6qi7Yp9PZGavrZR1-wtaCKvz2AWnnjjoSH19rhYG_E4L4MeUtlIm9kD3iqeisl6XCpwVyzGt3f6l-heFMaVNtOBFVRN7a2eykmTk-KGyR6Dt7VnHJsLD2poI3350g5o1bpbRdtdfpqIop55Uv8liiTN5lL7Bv/s1600/image1%20%284%29.png"/></a></div>
Last year we wrote about how <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">moving native code in Android from C++ to Rust has resulted in fewer security vulnerabilities</a>. Most of the components we mentioned then were system services in userspace (running under Linux), but these are not the only components typically written in memory-unsafe languages. Many security-critical components of an Android system run in a “bare-metal” environment, outside of the Linux kernel, and these are historically written in C. As part of our efforts to <a href="https://security.googleblog.com/2023/02/hardening-firmware-across-android.html">harden firmware on Android devices</a>, we are increasingly using Rust in these bare-metal environments too.
</p>
<p>
To that end, we have <a href="https://cs.android.com/android/platform/superproject/+/main:packages/modules/Virtualization/pvmfw/">rewritten the Android Virtualization Framework’s protected VM (pVM) firmware in Rust</a> to provide a memory safe foundation for the pVM root of trust. This firmware performs a similar function to a bootloader, and was initially built on top of <a href="https://github.com/u-boot/u-boot">U-Boot</a>, a widely used open source bootloader. However, U-Boot was not designed with security in a hostile environment in mind, and there have been <a href="https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&query=U-Boot&search_type=all&isCpeNameSearch=false">numerous security vulnerabilities</a> found in it due to out of bounds memory access, integer underflow and memory corruption. Its VirtIO drivers in particular had a number of <a href="https://lists.denx.de/pipermail/u-boot/2022-March/478466.html">missing</a> or <a href="https://lists.denx.de/pipermail/u-boot/2022-March/479626.html">problematic</a> bounds checks. We fixed the specific issues we found in U-Boot, but by leveraging Rust we can avoid these sorts of memory-safety vulnerabilities in future. The new Rust pVM firmware was released in Android 14.
</p>
<p>
As part of this effort, we contributed back to the Rust community by using and contributing to existing crates where possible, and publishing a number of new crates as well. For example, for VirtIO in pVM firmware we’ve spent time fixing bugs and soundness issues in the existing <a href="https://crates.io/crates/virtio-drivers">virtio-drivers</a> crate, as well as adding new functionality, and are now helping maintain this crate. We’ve published crates for <a href="https://crates.io/crates/smccc">making PSCI and other Arm SMCCC calls</a>, and for <a href="https://crates.io/crates/aarch64-paging">managing page tables</a>. These are just a start; we plan to release more Rust crates to support bare-metal programming on a range of platforms. These crates are also being used outside of Android, such as in <a href="https://github.com/project-oak/oak">Project Oak</a> and the <a href="https://google.github.io/comprehensive-rust/bare-metal.html">bare-metal section</a> of our <a href="https://security.googleblog.com/2023/09/scaling-rust-adoption-through-training.html">Comprehensive Rust</a> course.
</p>
<p>
<H2>Training engineers</H2>
</p>
<p>
Many engineers have been positively surprised by how productive and pleasant Rust is to work with, providing nice high-level features even in low-level environments. The engineers working on these projects come from a range of backgrounds. Our comprehensive Rust course has helped experienced and novice programmers quickly come up to speed. Anecdotally the Rust type system (including the borrow checker and lifetimes) helps avoid making mistakes that are easily made in C or C++, such as leaking pointers to stack-allocated values out of scope.
</p>
<p>
One of our bare-metal Rust course attendees had this to say:
</p>
<pre class="prettyprint">"types can be built that bring in all of Rust's niceties and safeties and
yet still compile down to extremely efficient code like writes
of constants to memory-mapped IO."
</pre>
<p>
97% of attendees that completed a survey agreed the course was worth their time.
</p>
<p>
<H2>Advantages and challenges</H2>
</p>
<p>
Device drivers are often written in an object-oriented fashion for flexibility, even in C. Rust traits, which can be seen as a form of compile-time polymorphism, provide a useful high-level abstraction for this. In many cases this can be resolved entirely at compile time, with no runtime overhead of dynamic dispatch via vtables or structs of function pointers.
</p>
<p>
There have been some challenges. Safe Rust’s type system is designed with an implicit assumption that the only memory the program needs to care about is allocated by the program (be it on the stack, the heap, or statically), and only used by the program. Bare-metal programs often have to deal with MMIO and shared memory, which break this assumption. This tends to require a lot of unsafe code and raw pointers, with limited tools for encapsulation. There is some disagreement in the Rust community about the soundness of references to MMIO space, and the facilities for working with raw pointers in stable Rust are currently somewhat limited. The stabilisation of <code><a href="https://github.com/rust-lang/rust/issues/106655">offset_of</a></code>, <code><a href="https://github.com/rust-lang/rust/issues/74265">slice_ptr_get</a></code>, <code><a href="https://github.com/rust-lang/rust/issues/71146">slice_ptr_len</a></code>, and other nightly features will improve this, but it is still challenging to encapsulate cleanly. Better syntax for accessing struct fields and array indices via raw pointers without creating references would also be helpful.
<p>
The concurrency introduced by interrupt and exception handlers can also be awkward, as they often need to access shared mutable state but can’t rely on being able to take locks. Better abstractions for critical sections will help somewhat, but there are some exceptions that can’t practically be disabled, such as page faults used to implement copy-on-write or other on-demand page mapping strategies.
</p>
<p>
Another issue we’ve had is that some unsafe operations, such as manipulating the page table, can’t be encapsulated cleanly as they have safety implications for the whole program. Usually in Rust we are able to encapsulate unsafe operations (operations which may cause undefined behaviour in some circumstances, because they have contracts which the compiler can’t check) in safe wrappers where we ensure the necessary preconditions so that it is not possible for any caller to cause undefined behaviour. However, mapping or unmapping pages in one part of the program can make other parts of the program invalid, so we haven’t found a way to provide a fully general safe interface to this. It should be noted that the same concerns apply to a program written in C, where the programmer always has to reason about the safety of the whole program.
</p>
<p>
Some people adopting Rust for bare-metal use cases have raised concerns about binary size. We have seen this in some cases; for example our Rust pVM firmware binary is around 460 kB compared to 220 kB for the earlier C version. However, this is not a fair comparison as we also added more functionality which allowed us to remove other components from the boot chain, so the overall size of all VM boot chain components was comparable. We also weren’t particularly optimizing for binary size in this case; speed and correctness were more important. In cases where binary size is critical, compiling with <a href="https://docs.rust-embedded.org/book/unsorted/speed-vs-size.html">size optimization</a>, being careful about dependencies, and avoiding Rust’s string formatting machinery in release builds usually allows comparable results to C.
</p>
<p>
Architectural support is another concern. Rust is generally <a href="https://doc.rust-lang.org/stable/rustc/platform-support.html">well supported</a> on the Arm and RISC-V cores that we see most often, but support for more esoteric architectures (for example, the Qualcomm Hexagon DSP included in many Qualcomm SoCs used in Android phones) can be lacking compared to C.
</p>
<p>
<H2>The future of bare-metal Rust</H2>
</p>
<p>
Overall, despite these challenges and limitations, we’ve still found Rust to be a significant improvement over C (or C++), both in terms of safety and productivity, in all the bare-metal use cases where we’ve tried it so far. We plan to use it wherever practical.
</p>
<p>
As well as the work in the Android Virtualization Framework, the team working on <a href="https://source.android.com/docs/security/features/trusty">Trusty</a> (the open-source Trusted Execution Environment used on Pixel phones, among others) have been hard at work adding support for Trusted Applications written in Rust. For example, the <a href="https://cs.android.com/android/platform/superproject/+/main:system/keymint/ta/">reference KeyMint Trusted Application implementation</a> is now in Rust. And there’s more to come in future Android devices, as we continue to use Rust to improve security of the devices you trust.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-19198633947225205522023-10-06T10:21:00.001-04:002023-10-06T10:21:05.040-04:00Expanding our exploit reward program to Chrome and Cloud<span class="byline-author">Stephen Roettger and Marios Pomonis, Google Software Engineers</span><div><br /></div><div><span id="docs-internal-guid-55f78d6e-7fff-e701-60e9-b961796c5a95"><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">In 2020, we launched a novel format for our vulnerability reward program (VRP) with the </span><a href="https://security.googleblog.com/2020/05/expanding-our-work-with-open-source.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">kCTF VRP</span></a><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and its continuation kernelCTF. For the first time, security researchers could get bounties for n-day exploits even if they didn’t find the vulnerability themselves. This format proved valuable in </span><a href="https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html" style="text-decoration-line: none;"><span style="color: #1155cc; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">improving our understanding</span></a><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> of the most widely exploited parts of the linux kernel. Its success motivated us to expand it to new areas and we're now excited to announce that we're extending it to two new targets: v8CTF and kvmCTF.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Today, we're launching v8CTF, a CTF focused on V8, the JavaScript engine that powers Chrome. kvmCTF is an upcoming CTF focused on Kernel-based Virtual Machine (KVM) that will be released later in the year.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">As with kernelCTF, we will be paying bounties for successful exploits against these platforms, n-days included. This is on top of any existing rewards for the vulnerabilities themselves. For example, if you find a vulnerability in V8 and then write an exploit for it, it can be eligible under both the Chrome VRP and the v8CTF.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We're always looking for ways to improve the security posture of our products, and we want to learn from the security community to understand how they will approach this challenge. If you're successful, you'll not only earn a reward, but you'll also help us make our products more secure for everyone. This is also a good opportunity to learn about technologies and gain hands-on experience exploiting them.</span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><br /><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; padding: 3pt 0pt 6pt 0pt;"><span style="background-color: transparent; color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Besides learning about exploitation techniques, we’ll also leverage this program to experiment with new mitigation ideas and see how they perform against real-world exploits. For mitigations, it’s crucial to assess their effectiveness early on in the process, and you can help us battle test them.</span></p><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; padding: 3pt 0pt 6pt 0pt;"><span style="background-color: transparent; color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt; padding: 3pt 0pt 6pt 0pt;"><span style="background-color: transparent; color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 6pt; margin-top: 0pt; padding: -3pt 0pt 0pt 0pt;"><span style="background-color: transparent; color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><b>How do I participate?</b></span></p><ul style="margin-bottom: 0; margin-top: 0; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 3pt;"><span style="background-color: transparent; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">First, make sure to check out the rules for </span><a href="https://github.com/google/security-research/blob/master/v8ctf/rules.md" style="text-decoration-line: none;"><span style="background-color: transparent; color: #1155cc; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">v8CTF</span></a><span style="background-color: transparent; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;"> or </span><a href="https://github.com/google/security-research/blob/master/kvmctf/rules.md" style="text-decoration-line: none;"><span style="background-color: transparent; color: #1155cc; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; text-wrap: wrap; vertical-align: baseline;">kvmCTF</span></a><span style="background-color: transparent; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">. This page contains up-to-date information about the types of exploits that are eligible for rewards, as well as the limits and restrictions that apply.</span></p></li><li aria-level="1" dir="ltr" style="color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="background-color: white; line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;"><span style="background-color: transparent; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Once you have identified a vulnerability present in our deployed version, exploit it, and grab the flag. It doesn’t even have to be an 0-day!</span></p></li><li aria-level="1" dir="ltr" style="color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="background-color: white; line-height: 1.38; margin-bottom: 6pt; margin-top: 0pt;"><span style="background-color: transparent; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Send us the flag by filling out the form linked in the rules and we’ll take it from there.</span></p></li></ul><div><span style="color: #1f1f1f; font-family: Google Sans, sans-serif;"><span style="font-size: 14px; white-space-collapse: preserve;"><br /></span></span></div><div><span style="color: #1f1f1f; font-family: Google Sans, sans-serif;"><span style="font-size: 14px; white-space-collapse: preserve;"><br /></span></span></div><div><span style="color: #1f1f1f; font-family: Google Sans, sans-serif;"><span style="font-size: 14px; white-space-collapse: preserve;"><br /></span></span></div><p dir="ltr" style="background-color: white; line-height: 1.38; margin-bottom: 6pt; margin-top: 6pt;"><span style="background-color: transparent; color: #1f1f1f; font-family: "Google Sans", sans-serif; font-size: 10.5pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We're looking forward to seeing what you can find!</span></p></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-49123487810174543562023-09-27T12:50:00.003-04:002023-09-27T13:03:56.013-04:00SMS Security & Privacy Gaps Make It Clear Users Need a Messaging Upgrade<span class="byline-author">Posted by Eugene Liderman and Roger Piqueras Jover </span>
<p>
SMS texting is frozen in time.
</p>
<p>
People still use and rely on trillions of SMS texts each year to exchange messages with friends, share family photos, and copy two-factor authentication codes to access sensitive data in their bank accounts. It’s hard to believe that at a time where technologies like AI are transforming our world, a forty-year old mobile messaging standard is still so prevalent.
</p>
<p>
Like any forty-year-old technology, SMS is antiquated compared to its modern counterparts. That’s especially concerning when it comes to security.
</p>
<p>
<br><strong>The World Has Changed, But SMS Hasn’t Changed With It</strong>
</p>
<p>
According to a recent <a href="https://www.dekra.com/en/exploring-sms-security-download/">whitepaper</a> from Dekra, a safety certifications and testing lab, the security shortcomings of SMS can notably lead to:
</p>
<ul>
<li><strong>SMS Interception:</strong> Attackers can intercept SMS messages by exploiting vulnerabilities in mobile carrier networks. This can allow them to read the contents of SMS messages, including sensitive information such as two-factor authentication codes, passwords, and credit card numbers due to the lack of encryption offered by SMS.
<li><strong>SMS Spoofing:</strong> Attackers can spoof SMS messages to launch phishing attacks to make it appear as if they are from a legitimate sender. This can be used to trick users into clicking on malicious links or revealing sensitive information. And because carrier networks have independently developed their approaches to deploying SMS texts over the years, the inability for carriers to exchange reputation signals to help identify fraudulent messages has made it tough to detect spoofed senders distributing potentially malicious messages.
</li>
</ul>
<p>
These findings add to the <a href="https://cacm.acm.org/magazines/2020/12/248798-security-analysis-of-sms-as-a-second-factor-of-authentication/fulltext">well-established facts about SMS’ weaknesses</a>, lack of encryption chief among them.<br>
</p>
<p>
Dekra also compared SMS against a modern secure messaging protocol and found it lacked any built-in security functionality. <br><br>According to Dekra, SMS users can’t answer ‘yes’ to any of the following basic security questions: <br>
</p>
<ul>
<li><strong>Confidentiality: </strong>Can I trust that no one else can read my SMSs?
<li><strong>Integrity:</strong> Can I trust that the content of the SMS that I receive is not modified?
<li><strong>Authentication: </strong>Can I trust the identity of the sender of the SMS that I receive?
</li>
</ul>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ3UjJPX7qXLeKJ8ogXklnjaIxMhBk9MQwqBvsRhdwpNCq0Evb0x3g5cFnMkrYSh69O8dZsyuJUpxES9ckK9pmeEbu46CkpSYz5qhijfny7yGIspeyJSrpcgEeelA3isf-BI0Rg5e8QtuQek_MgfUT3UI7NLJpffjd_0Sc9nws1ZWPISwDJuWF-JG1euVk/s1600/Screenshot%202023-09-27%209.38.45%20AM.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="191" data-original-width="702" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQ3UjJPX7qXLeKJ8ogXklnjaIxMhBk9MQwqBvsRhdwpNCq0Evb0x3g5cFnMkrYSh69O8dZsyuJUpxES9ckK9pmeEbu46CkpSYz5qhijfny7yGIspeyJSrpcgEeelA3isf-BI0Rg5e8QtuQek_MgfUT3UI7NLJpffjd_0Sc9nws1ZWPISwDJuWF-JG1euVk/s1600/Screenshot%202023-09-27%209.38.45%20AM.png"/></a></div>
<p>
But this isn’t just theoretical: cybercriminals have also caught on to the lack of security protections SMS provides and have repeatedly exploited its weakness. Both novice hackers and advanced threat actor groups (such as <a href="https://www.mandiant.com/resources/blog/unc3944-sms-phishing-sim-swapping-ransomware">UNC3944 / Scattered Spider </a>and <a href="https://www.mandiant.com/resources/blog/messagetap-who-is-reading-your-text-messages">APT41</a> investigated by <a href="https://www.mandiant.com/">Mandiant</a>, part of Google Cloud) leverage the security deficiencies in SMS to launch different types of attacks against users and corporations alike.
</p>
<p>
Malicious cyber attacks that exploit the insecurity of SMS have resulted in identity theft, personal or corporate financial losses, unauthorized access to accounts and services, and worse.<br><br><strong>Users Care About Messaging Security and Privacy Now More Than Ever </strong>
</p>
<p>
Both iOS and Android users understand the importance of security and privacy when sending and receiving messages, and now, they want more protection than what SMS can provide.
</p>
<p>
A new <a href="https://today.yougov.com/">YouGov</a> study examined how device users across platforms think and feel about SMS texting as well as their desire for more security to protect their text messages.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6KPK3ASID0db94i88ArIsRtL5P20CiVujlJdSpzqYy0IeLY-V-l43XVcqt_1x5S3dxQCNxg2S02Y5FzBt_cLofmvbNlum-ly0wWR3ZD55MjNBWB5auJoRfjxPUdE5yXzzPD69nLUSNJygcL0ObaEjUcSqVwI0z61cUzs9hUajeRbogbxNSWYSq-I8QdVF/s1600/Screenshot%202023-09-27%2010.03.39%20AM.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="831" data-original-width="515" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6KPK3ASID0db94i88ArIsRtL5P20CiVujlJdSpzqYy0IeLY-V-l43XVcqt_1x5S3dxQCNxg2S02Y5FzBt_cLofmvbNlum-ly0wWR3ZD55MjNBWB5auJoRfjxPUdE5yXzzPD69nLUSNJygcL0ObaEjUcSqVwI0z61cUzs9hUajeRbogbxNSWYSq-I8QdVF/s1600/Screenshot%202023-09-27%2010.03.39%20AM.png"/></a></div>
</p>
<p>
<strong>It’s Time to Move on From SMS</strong>
</p>
<p>
<br>The security landscape as it relates to SMS is simple:<br>
</p>
<ul>
<li>SMS is widely used
<li>SMS is easily abused because it has so few protections
<li>Smartphone users across mobile platforms care more about security than ever before
</li>
</ul>
<p>
The continued evolution of the mobile ecosystem will depend on users' ability to trust and feel safe, regardless of the phone they may be using. The security of the mobile ecosystem is only as strong as its weakest link and, unfortunately, SMS texting is both a large and weak link in the chain largely because texts between iPhones and Androids revert to SMS.
</p>
<p>
As a mobile ecosystem, we collectively owe it to all users, across platforms, to enable them to be as safe as possible. It’s a shame that a problem like texting security remains as prominent as it is, particularly when new protocols like <a href="https://www.gsma.com/futurenetworks/rcs/the-rcs-ecosystem/">RCS</a> are well-established and would drastically improve security for everyone. <br><br>Today, most global carriers and over 500 Android device manufacturers already support RCS and RCS is <a href="https://support.google.com/messages/thread/229405182/your-rcs-conversations-are-now-fully-end-to-end-encrypted?hl=en&sjid=6630133419705040743-NA">enabled by default on Messages by Google</a>. However, whether the solution is RCS or something else, it’s important that our industry moves towards a solution to a problem that should have been fixed before the smartphone era ever began.
</p>
<hr>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-70422887815679006912023-09-21T12:00:00.003-04:002023-09-21T12:00:57.668-04:00Scaling Rust Adoption Through Training<span class="byline-author">Posted by Martin Geisler, Android team</span>
<p>
Android 14 is the third major Android release with Rust support. We are already seeing a number of benefits:
</p>
<ul>
<li><em>Productivity:</em> Developers <a href="https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html">quickly feel productive writing Rust</a>. They report important indicators of development velocity, such as confidence in the code quality and ease of code review.
<li><em>Security:</em> There has been a <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">reduction in memory safety vulnerabilities</a> as we shift more development to memory safe languages.
</li>
</ul>
<p>
These positive early results provided an enticing motivation to increase the speed and scope of Rust adoption. We hoped to accomplish this by investing heavily in training to expand from the early adopters.
</p>
<h1>Scaling up from Early Adopters</h1>
<p>
Early adopters are often willing to accept more risk to try out a new technology. They know there will be some inconveniences and a steep learning curve but are willing to learn, often on their own time.
</p>
<p>
Scaling up Rust adoption required moving beyond early adopters. For that we need to ensure a baseline level of comfort and productivity within a set period of time. An important part of our strategy for accomplishing this was training. Unfortunately, the type of training we wanted to provide simply didn’t exist. We made the decision to write and implement our own Rust training.
</p>
<h1>Training Engineers</h1>
<p>
Our goals for the training were to:
</p>
<ul>
<li><em>Quickly ramp up engineers:</em> It is hard to take people away from their regular work for a long period of time, so we aimed to provide a solid foundation for using Rust in days, not weeks. We could not make anybody a Rust expert in so little time, but we could give people the tools and foundation needed to be productive while they continued to grow. The goal is to enable people to use Rust to be productive members of their teams. The time constraints meant we couldn’t teach people programming from scratch; we also decided not to teach macros or unsafe Rust in detail.
<li><em>Make it engaging (and fun!):</em> We wanted people to see a lot of Rust while also getting hands-on experience. Given the scope and time constraints mentioned above, the training was necessarily information-dense. This called for an interactive setting where people could quickly ask questions to the instructor. Research shows that retention improves when people can quickly verify assumptions and practice new concepts.
<li><em>Make it relevant for Android:</em> The Android-specific tooling for Rust was <a href="https://source.android.com/docs/setup/build/rust/building-rust-modules/overview">already documented</a>, but we wanted to show engineers how to use it via worked examples. We also wanted to document emerging standards, such as using <a href="https://docs.rs/thiserror">thiserror</a> and <a href="https://docs.rs/anyhow">anyhow</a> crates for error handling. Finally, because Rust is a new language in the Android Platform (AOSP), we needed to show how to interoperate with existing languages such as Java and C++.
</li>
</ul>
<p>
With those three goals as a starting point, we looked at the existing material and available tools.
</p>
<h3>Existing Material</h3>
<p>
Documentation is a key value of the Rust community and there are many great resources available for learning Rust. First, there is the freely available <a href="https://doc.rust-lang.org/book/">Rust Book</a>, which covers almost all of the language. Second, the standard library is <a href="https://doc.rust-lang.org/std/">extensively documented</a>.
</p>
<p>
Because we knew our target audience, we could make stronger assumptions than most material found online. We created the course for engineers with at least 2–3 years of coding experience in either C, C++, or Java. This allowed us to move quickly when explaining concepts familiar to our audience, such as "control flow", “stack vs heap”, and “methods”. People with other backgrounds can learn Rust from the many <a href="https://google.github.io/comprehensive-rust/other-resources.html">other resources</a> freely available online.
</p>
<h3>Technology</h3>
<p>
For free-form documentation, <a href="https://rust-lang.github.io/mdBook/">mdBook</a> has become the de facto standard in the Rust community. It is used for official documentation such as the <a href="https://doc.rust-lang.org/book/">Rust Book</a> and <a href="https://doc.rust-lang.org/reference">Rust Reference</a>.
</p>
<p>
A particularly interesting feature is the ability to embed executable snippets of Rust code. This is key to making the training engaging since the code can be edited live and executed directly in the slides:
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitWr0cX32U1kCjd473gOytAHEDO4H56wO60-cWYNzJTAAFmmVjjHrXWUnSUkN21CWYrqPpqj8mKk6Uz1VXRw-BAJrTBNaYqCe948Fn2HRU0myoXzmElX0kT82x5hWbmt9nIz9_x2Gsw3JrVm5F6T1ZWbu_PoxZ1qMz7nIlxq5o7tAs2CkFfjJt8quMEqnu/s1600/image1%20%282%29.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="146" data-original-width="648" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitWr0cX32U1kCjd473gOytAHEDO4H56wO60-cWYNzJTAAFmmVjjHrXWUnSUkN21CWYrqPpqj8mKk6Uz1VXRw-BAJrTBNaYqCe948Fn2HRU0myoXzmElX0kT82x5hWbmt9nIz9_x2Gsw3JrVm5F6T1ZWbu_PoxZ1qMz7nIlxq5o7tAs2CkFfjJt8quMEqnu/s1600/image1%20%282%29.png"/></a></div>
<p>
In addition to being a familiar community standard, mdBook offers the following important features:
</p>
<ul>
<li><em>Maintainability:</em> <code>mdbook test</code> compiles and executes every code snippet in the course. This allowed us to evolve the class over time while ensuring that we always showed valid code to the participants.
<li><em>Extensibility:</em> mdBook has a <a href="https://rust-lang.github.io/mdBook/for_developers/index.html">plugin system</a> which allowed us to extend the tool as needed. We relied on this feature for translations and <a href="https://github.com/boozook/mdbook-svgbob">ASCII art diagrams</a>.
</li>
</ul>
<p>
These features made it easy for us to choose mdBook. While mdBook is not designed for presentations, the output looked OK on a projector when we limited the vertical size of each page.
</p>
<h3>Supporting Translations</h3>
<p>
Android has developers and OEM partners in many countries. It is critical that they can adapt existing Rust code in AOSP to fit their needs. To support translations, we developed <a href="https://github.com/google/mdbook-i18n-helpers">mdbook-i18n-helpers</a>. Support for <a href="https://github.com/rust-lang/mdBook/issues/5">multilingual documentation</a> has been a community wish since 2015 and we are glad to see the plugins being adopted by several other projects to produce maintainable multilingual documentation for everybody.
</p>
<h1>Comprehensive Rust</h1>
<p>
With the technology and format nailed down, we started writing the course. We roughly followed the outline from the Rust Book since it covered most of what we need to cover. This gave us a three day course which we called <a href="https://google.github.io/comprehensive-rust/welcome-day-1.html">Rust Fundamentals</a>. We designed it to run for three days for five hours a day and encompass Rust syntax, semantics, and important concepts such as traits, generics, and error handling.
</p>
<p>
We then extended Rust Fundamentals with three deep dives:
</p>
<ul>
<li><em><a href="https://google.github.io/comprehensive-rust/android.html">Rust in Android</a>:</em> a half-day course on using Rust for AOSP development. It includes interoperability with C, C++, and Java.
<li><em><a href="https://google.github.io/comprehensive-rust/bare-metal.html">Bare-metal Rust</a>:</em> a full-day class on using Rust for bare-metal development. Android devices ship significant amounts of firmware. These components are often foundational in nature (for example, the bootloader, which establishes the trust for the rest of the system), thus they must be secure.
<li><em><a href="https://google.github.io/comprehensive-rust/concurrency.html">Concurrency in Rust</a>:</em> a full-day class on concurrency in Rust. We cover both multithreading with blocking synchronization primitives (such as mutexes) and async/await concurrency (cooperative multitasking using futures).
</li>
</ul>
<p>
A large set of in-house and community translators have helped translate the course into several languages. The full translations were <a href="https://google.github.io/comprehensive-rust/pt-BR/">Brazilian Portuguese</a> and <a href="https://google.github.io/comprehensive-rust/ko/">Korean</a>. We are working on <a href="https://google.github.io/comprehensive-rust/zh-CN/">Simplified Chinese</a> and <a href="https://google.github.io/comprehensive-rust/zh-TW/">Traditional Chinese</a> translations as well.
</p>
<h3>Course Reception</h3>
<p>
We started teaching the class in late 2022. In 2023, we hired a vendor, <a href="https://immunant.com/">Immunant</a>, to teach the majority of classes for Android engineers. This was important for scalability and for quality: dedicated instructors soon discovered where the course participants struggled and could adapt the delivery. In addition, over 30 Googlers have taught the course worldwide.
</p>
<p>
More than 500 Google engineers have taken the class. Feedback has been very positive: 96% of participants agreed it was worth their time. People consistently told us that they loved the interactive style, highlighting how it helped to be able to ask clarifying questions at any time. Instructors noted that people gave the course their undivided attention once they realized it was live. Live-coding demands a lot from the instructor, but it is worth it due to the high engagement it achieves.
</p>
<p>
Most importantly, people exited this course and were able to be immediately productive with Rust in their day jobs. When participants were asked three months later, they confirmed that they were able to write and review Rust code. This matched the results from the <a href="https://opensource.googleblog.com/2023/06/rust-fact-vs-fiction-5-insights-from-googles-rust-journey-2022.html">much larger survey we made in 2022</a>.
</p>
<h1>Looking Forward</h1>
<p>
We have been teaching Rust classes at Google for a year now. There are a few things that we want to improve: <a href="https://github.com/google/comprehensive-rust/issues/510">better topic ordering</a>, <a href="https://github.com/google/comprehensive-rust/issues/1082">more exercises</a>, and <a href="https://github.com/google/comprehensive-rust/issues/1083">more speaker notes</a>. We would also like to extend the course with <a href="https://github.com/google/comprehensive-rust/discussions/32">more deep dives</a>. Pull requests are very welcome!
</p>
<p>
The full course is available for free at <a href="https://google.github.io/comprehensive-rust/">https://google.github.io/comprehensive-rust/</a>. We are thrilled to see people starting to use Comprehensive Rust for classes around the world. We hope it can be a useful resource for the Rust community and that it will help both small and large teams get started on their Rust journey!
</p>
<h1>Thanks!</h1>
<p>
We are grateful to the <a href="https://github.com/google/comprehensive-rust/graphs/contributors">190+ contributors</a> from all over the world who created more than 1,000 pull requests and issues on GitHub. Their bug reports, fixes, and feedback improved the course in countless ways. This includes the 50+ people who worked hard on writing and maintaining the many translations.
</p>
<p>
Special thanks to <a href="https://github.com/qwandor">Andrew Walbran</a> for writing Bare-metal Rust and to <a href="https://github.com/rbehjati">Razieh Behjati</a>, <a href="https://github.com/djmitche">Dustin Mitchell</a>, and <a href="https://github.com/sakex">Alexandre Senges</a> for writing Concurrency in Rust.
</p>
<p>
We also owe a great deal of thanks to the many volunteer instructors at Google who have been spending their time teaching classes around the globe. Your feedback has helped shape the course.
</p>
<p>
Finally, thanks to Jeffrey Vander Stoep, Ivan Lozano, Matthew Maurer, Dmytro Hrybenko, and Lars Bergstrom for providing feedback on this post.
</p>Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-24866406894661745762023-09-15T14:10:00.002-04:002023-09-15T14:11:38.953-04:00Capslock: What is your code really capable of?<span class="byline-author">Jess McClintock and John Dethridge, Google Open Source Security Team, and Damien Miller, Enterprise Infrastructure Protection Team</span><div><br /></div><div><span id="docs-internal-guid-2be4e047-7fff-59e2-e91c-3d9d06c0694a"><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">When you import a third party library, do you review every line of code? Most software packages depend on external libraries, trusting that those packages aren’t doing anything unexpected. If that trust is violated, the consequences can be huge—regardless of whether the package is malicious, or well-intended but using overly broad permissions, such as with</span><a href="https://security.googleblog.com/2021/12/understanding-impact-of-apache-log4j.html" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;"> Log4j in 2021.</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> Supply chain security is a growing issue, and we hope that greater transparency into package capabilities will help make secure coding easier for everyone.</span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Avoiding bad dependencies can be hard without appropriate information on what the dependency’s code actually does, and reviewing every line of that code is an immense task. Every dependency also brings its own dependencies, compounding the need for review across an expanding web of transitive dependencies. But what if there was an easy way to know the capabilities–the privileged operations accessed by the code–of your dependencies? </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Capslock is a capability analysis CLI tool that informs users of privileged operations (like network access and arbitrary code execution) in a given package and its dependencies. Last month </span><a href="https://github.com/google/capslock" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">we published</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> the alpha version of Capslock for the Go language, which can analyze and report on the capabilities that are used beneath the surface of open source software. </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">This CLI tool will provide deeper insights into the behavior of dependencies by reporting code paths that access privileged operations in the standard libraries. In upcoming versions we will add support for open source maintainers to prescribe and sandbox the capabilities required for their packages, highlighting to users what capabilities are present and alerting them if they change.</span></p><br /><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></h2><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></h2><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Capabilities vs Vulnerabilities</span></h2><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Vulnerability management is an important part of your supply chain security, but it doesn’t give you a full picture of whether your dependencies are safe to use. Adding capability analysis into your security posture, gives you a better idea of the types of behavior you can expect from your dependencies, identifies potential weak points, and allows you to make a more informed choice about using a given dependency. </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Capslock is motivated by the belief that the principle of least privilege—the idea that access should be limited to the minimal set that is feasible and practical—should be a first-class design concept for secure and usable software. Applied to software development, this means that a package should be allowed access only to the capabilities that it requires as part of its core behaviors. For example, you wouldn’t expect a data analysis package to need access to the network or a logging library to include remote code execution capabilities. </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Capslock is initially rolling out for Go, a language with a </span><a href="https://go.dev/blog/13years" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">strong security commitment</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> and </span><a href="https://security.googleblog.com/2023/04/supply-chain-security-for-go-part-1.html" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">fantastic tooling</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> for finding known vulnerabilities in package dependencies. When Capslock is used alongside Go’s vulnerability management tools, developers can use the additional, complementary signals to inform how they interpret vulnerabilities in their dependencies. </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">These capability signals can be used to</span></p><br /><ul style="margin-bottom: 0px; margin-top: 0px; padding-inline-start: 48px;"><li aria-level="1" dir="ltr" style="color: #3c4043; font-family: "Google Sans", sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Find code with the highest levels of access to prioritize audits, code reviews and vulnerability patches</span></p></li><li aria-level="1" dir="ltr" style="color: #3c4043; font-family: "Google Sans", sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Compare potential dependencies, or look for alternative packages when an existing dependency is no longer appropriate</span></p></li><li aria-level="1" dir="ltr" style="color: #3c4043; font-family: "Google Sans", sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Surface unwanted capability usage in packages to uncover new vulnerabilities or identify supply chain attacks in progress</span></p></li><li aria-level="1" dir="ltr" style="color: #3c4043; font-family: "Google Sans", sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Monitor for unexpected emerging capabilities due to package version or dependency changes, and even integrate capability monitoring into CI/CD pipelines </span></p></li><li aria-level="1" dir="ltr" style="color: #3c4043; font-family: "Google Sans", sans-serif; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; list-style-type: disc; vertical-align: baseline; white-space: pre;"><p dir="ltr" role="presentation" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span style="font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-wrap: wrap; vertical-align: baseline;">Filter vulnerability data to respond to the most relevant cases, such as finding packages with network access during a network-specific vulnerability alert </span></p></li></ul><br /><br /><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></h2><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></h2><h2 dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Using Capslock</span></h2><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB9cWg2UwnDkVwJ0KjayewXc_UrYxS1dkspKW01MDm8CXQaNC8EUkP3bHEjvN1ogFc0zs5CIKGw13bbVHibs76-i4IbSTGd-SF8TCV9U9GR8CC4w3GVeiV_mQMsxenC6zaee7YEG6VrDQ9zX_d53RHsO5VuuEUyoq2FbrgVgw9w6-hieDeKJtN62H9Q_tc/s1600/pasted%20image%200.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1081" data-original-width="1600" height="371" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiB9cWg2UwnDkVwJ0KjayewXc_UrYxS1dkspKW01MDm8CXQaNC8EUkP3bHEjvN1ogFc0zs5CIKGw13bbVHibs76-i4IbSTGd-SF8TCV9U9GR8CC4w3GVeiV_mQMsxenC6zaee7YEG6VrDQ9zX_d53RHsO5VuuEUyoq2FbrgVgw9w6-hieDeKJtN62H9Q_tc/w549-h371/pasted%20image%200.png" width="549" /></a></div><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><br /></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We are looking forward to adding new features in future releases, such as better support for declaring the expected capabilities of a package, and extending to other programming languages. We are working to apply Capslock at scale and make capability information for open source packages broadly available in various community tools like </span><a href="https://deps.dev/" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">deps.dev</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">. </span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">You can </span><a href="https://github.com/google/capslock" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">try Capslock now</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">, and we hope you find it useful for auditing your external dependencies and making informed decisions on your code’s capabilities.</span></p><br /><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><br /></span></p><p dir="ltr" style="line-height: 1.2; margin-bottom: 0pt; margin-top: 0pt;"><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">We’ll be at </span><a href="https://www.gophercon.com/agenda/session/1160360" style="text-decoration-line: none;"><span face=""Google Sans", sans-serif" style="color: #1155cc; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">Gophercon in San Diego on Sept 27th, 2023</span></a><span face=""Google Sans", sans-serif" style="color: #3c4043; font-size: 12pt; font-variant-alternates: normal; font-variant-east-asian: normal; font-variant-numeric: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">—come and chat with us! </span></p><br /><br /><br /></span></div>Kimberly Samrahttp://www.blogger.com/profile/17209446531010386292noreply@blogger.com0tag:blogger.com,1999:blog-1176949257541686127.post-25450258599595664472023-08-29T12:06:00.006-04:002023-11-01T13:50:53.222-04:00Android Goes All-in on Fuzzing<span class="byline-author">Posted by Hamzeh Zawawy and Jon Bottarini, Android Security</span>
<p>
Fuzzing is an effective technique for finding software vulnerabilities. Over the past few years Android has been focused on improving the effectiveness, scope, and convenience of fuzzing across the organization. This effort has directly resulted in improved test coverage, fewer security/stability bugs, and higher code quality. Our implementation of continuous fuzzing allows software teams to find new bugs/vulnerabilities, and prevent regressions automatically without having to manually initiate fuzzing runs themselves. This post recounts a brief history of fuzzing on Android, shares how Google performs fuzzing at scale, and documents our experience, challenges, and success in building an infrastructure for automating fuzzing across Android. If you’re interested in contributing to fuzzing on Android, we’ve included instructions on how to get started, and information on how Android’s VRP rewards fuzzing contributions that find vulnerabilities.
</p>
<h3>A Brief History of Android Fuzzing</h3>
<p>
Fuzzing has been around for many years, and Android was among the early large software projects to automate fuzzing and prioritize it similarly to unit testing as part of the broader goal to make Android the most secure and stable operating system. In 2019 Android kicked off the fuzzing project, with the goal to help institutionalize fuzzing by making it seamless and part of code submission. The Android fuzzing project resulted in an infrastructure consisting of Pixel phones and Google cloud based virtual devices that enabled scalable fuzzing capabilities across the entire Android ecosystem. This project has since grown to become the official internal fuzzing infrastructure for Android and performs thousands of fuzzing hours per day across hundreds of fuzzers.
</p>
<h3>Under the Hood: How Is Android Fuzzed</h3>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ-gVMdtX4fCzXHXKww5a-v6ZvrPUo9jAKM0bJzk7okJ5o8-psXKVclHrFV5qF5WKk94fcg5WW7B_NxBfZMlJQ8MrRAoPAOTyml9gn3ZxEl1gR27eoV1TsftRxozM6XkuxvNO6Bh25lq_MFOZTrJZ0-qoyAy3fMpMGV5j7v2eaTemE9_Mttxuqk9pVpnwH/s1600/image7.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="690" data-original-width="1999" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ-gVMdtX4fCzXHXKww5a-v6ZvrPUo9jAKM0bJzk7okJ5o8-psXKVclHrFV5qF5WKk94fcg5WW7B_NxBfZMlJQ8MrRAoPAOTyml9gn3ZxEl1gR27eoV1TsftRxozM6XkuxvNO6Bh25lq_MFOZTrJZ0-qoyAy3fMpMGV5j7v2eaTemE9_Mttxuqk9pVpnwH/s1600/image7.png"/></a></div>
<p>
<strong>Step 1: Define and find all the fuzzers in Android repo</strong>
</p>
<p>
The first step is to integrate fuzzing into the Android build system (<a href="https://cs.android.com/android/platform/superproject/+/master:build/soong/cc/fuzz.go">Soong</a>) to enable build fuzzer binaries. While developers are busy adding features to their codebase, they can include a fuzzer to fuzz their code and submit the fuzzer alongside the code they have developed. Android Fuzzing uses a build rule called <em>cc_fuzz</em> (see example below). cc_fuzz (we also support rust_fuzz and java_fuzz) defines a Soong module with source file(s) and dependencies that can be built into a binary.
</p>
<pre class="prettyprint">cc_fuzz {
name: "fuzzer_foo",
srcs: [
"fuzzer_foo.cpp",
],
static_libs: [
"libfoo",
],
host_supported: true,
}</pre>
<p>
A packaging rule in Soong finds all of these cc_fuzz definitions and builds them automatically. The actual fuzzer structure itself is very simple and consists of one main method (LLVMTestOneInput):
</p>
<pre class="prettyprint">#include <stddef.h>
#include <stdint.h>
extern "C" int LLVMFuzzerTestOneInput(
const uint8_t *data,
size_t size) {
// Here you invoke the code to be fuzzed.
return 0;
}</pre>
<p>
This fuzzer gets automatically built into a binary and along with its static/dynamic dependencies (as specified in the Android build file) are packaged into a zip file which gets added to the main zip containing all fuzzers as shown in the example below.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguO-0zvc3mrPKvS3FbLVk_RAiQvVkL68Y_X2aKbUCio-8w1ViyBzTN8mC9tE2oUgtUxc_WgVB-pTTlvexxxaUJoGfNQd7opTUFQ6XJg_S6MXljLtDiFmV4SsQY69s6cEsv7P-tIZUnZINKpcGlGA9VVLLPoYR4tX_1r99Ehx4XTcQQ4Ohe_XR-_rJFZCr0/s1600/image3.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="340" data-original-width="903" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEguO-0zvc3mrPKvS3FbLVk_RAiQvVkL68Y_X2aKbUCio-8w1ViyBzTN8mC9tE2oUgtUxc_WgVB-pTTlvexxxaUJoGfNQd7opTUFQ6XJg_S6MXljLtDiFmV4SsQY69s6cEsv7P-tIZUnZINKpcGlGA9VVLLPoYR4tX_1r99Ehx4XTcQQ4Ohe_XR-_rJFZCr0/s1600/image3.png"/></a></div>
<p>
</p>
<p>
<strong>Step 2: Ingest all fuzzers into Android builds</strong>
</p>
<p>
Once the fuzzers are found in the Android repository and they are built into binaries, the next step is to upload them to the cloud storage in preparation to run them on our backend. This process is run multiple times daily. The Android fuzzing infrastructure uses an open source continuous fuzzing framework (<a href="https://github.com/google/clusterfuzz">Clusterfuzz</a>) to run fuzzers continuously on Android devices and emulators. In order to run the fuzzers on clusterfuzz, the fuzzers zip files are renamed after the build and the latest build gets to run (see diagram below):
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLt2yxeoy_X24HnBK9zNZcmDnfLoafeqFq6h_h5L8DLPv0DvK1-ygU_uBWVGsD5YC60RATo_hmfXIezDYKwddWF06D7DCqt-bqFVBe14lTybWRtTk3r0rsmz9R-c5dbQHPULQzg8DoH5tZh8p08PvVFtFe9F-76EmfCTgH5jfQe3RnyZGaW56e5DBtIlnK/s1600/image1%20%281%29.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="259" data-original-width="842" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiLt2yxeoy_X24HnBK9zNZcmDnfLoafeqFq6h_h5L8DLPv0DvK1-ygU_uBWVGsD5YC60RATo_hmfXIezDYKwddWF06D7DCqt-bqFVBe14lTybWRtTk3r0rsmz9R-c5dbQHPULQzg8DoH5tZh8p08PvVFtFe9F-76EmfCTgH5jfQe3RnyZGaW56e5DBtIlnK/s1600/image1%20%281%29.png"/></a></div>
<p>
The fuzzer zip file contains the fuzzer binary, corresponding dictionary as well as a subfolder containing its dependencies and the git revision numbers (sourcemap) corresponding to the build. Sourcemaps are used to enhance stack traces and produce crash reports.
</p>
<p>
<strong>Step 3: Run fuzzers continuously and find bugs</strong>
</p>
<p>
Running fuzzers continuously is done through scheduled jobs where each job is associated with a set of physical devices or emulators. A job is also backed by a queue that represents the fuzzing tasks that need to be run. These tasks are a combination of running a fuzzer, reproducing a crash found in an earlier fuzzing run, or minimizing the corpus, among other tasks.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieKLHyEExuNbOgg_iUt8lQmf1zJU6qErfENrA-W2M3fGUAfgmKEpCcck6xDDL0_QRZ5KwbAdnF2cTqKvkkfcQ3UqPFjkWcQGWmxIDXBhEyxQpV9KBxPkEDnd7pL1vNaz_R9AWNZRLLFY7f8Umc96z4HwbeaDkaOoA4MHJmam2uYeYeKesI3B2H6YqBil9o/s1600/image6.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="534" data-original-width="1999" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieKLHyEExuNbOgg_iUt8lQmf1zJU6qErfENrA-W2M3fGUAfgmKEpCcck6xDDL0_QRZ5KwbAdnF2cTqKvkkfcQ3UqPFjkWcQGWmxIDXBhEyxQpV9KBxPkEDnd7pL1vNaz_R9AWNZRLLFY7f8Umc96z4HwbeaDkaOoA4MHJmam2uYeYeKesI3B2H6YqBil9o/s1600/image6.png"/></a></div>
<p>
Each fuzzer is run for multiple hours, or until they find a crash. After the run, Android fuzzing takes all of the interesting input discovered during the run and adds it to the fuzzer corpus. This corpus is then shared across fuzzer runs and grows over time. The fuzzer is then prioritized in subsequent runs according to the growth of new coverage and crashes found (if any). This ensures we provide the most effective fuzzers more time to run and find interesting crashes.
</p>
<p>
<strong>Step 4: Generate fuzzers line coverage</strong>
</p>
<p>
What good is a fuzzer if it’s not fuzzing the code you care about? To improve the quality of the fuzzer and to monitor the overall progress of Android fuzzing, two types of coverage metrics are calculated and available to Android developers. The first metric is for <a href="https://llvm.org/docs/LibFuzzer.html">edge</a> coverage which refers to edges in the Control Flow Graph (CFG). By instrumenting the fuzzer and the code being fuzzed, the fuzzing engine can track small snippets of code that get triggered every time execution flow reaches them. That way, fuzzing engines know exactly how many (and how many times) each of these instrumentation points got hit on every run so they can aggregate them and calculate the coverage.
</p>
<pre class="prettyprint">INFO: Seed: 2859304549
INFO: Loaded 1 modules (773 inline 8-bit counters): 773 [0x5610921000, 0x5610921305),
INFO: Loaded 1 PC tables (773 PCs): 773 [0x5610921308,0x5610924358),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2 INITED cov: 2 ft: 2 corp: 1/1b lim: 4 exec/s: 0 rss: 24Mb
#413 NEW cov: 3 ft: 3 corp: 2/9b lim: 8 exec/s: 0 rss: 24Mb L: 8/8 MS: 1 InsertRepeatedBytes-
#3829 NEW cov: 4 ft: 4 corp: 3/17b lim: 38 exec/s: 0 rss: 24Mb L: 8/8 MS: 1 ChangeBinInt-
...</pre>
<p>
Line coverage inserts instrumentation points specifying lines in the source code. Line coverage is very useful for developers as they can pinpoint areas in the code that are not covered and update their fuzzers accordingly to hit those areas in future fuzzing runs.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-MtfgWtNgQGeHeZJp_mwRlrnMaPwyA2oimGu-AAvwXRO7Mbe8KvAoXFhyKjUp_Iw0iB4zi5WosyXo8dFN-r4-WBONLjvro2L_YRRWEs-EVdP2IbJIwKECxsE12O-i5rXPGROEb-dSisATdb9YyjIwf_pFcCNNF9OYErUOuQ8Kw4_LmSUjfFZYtB5089K3/s1600/image4.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="320" data-original-width="1323" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-MtfgWtNgQGeHeZJp_mwRlrnMaPwyA2oimGu-AAvwXRO7Mbe8KvAoXFhyKjUp_Iw0iB4zi5WosyXo8dFN-r4-WBONLjvro2L_YRRWEs-EVdP2IbJIwKECxsE12O-i5rXPGROEb-dSisATdb9YyjIwf_pFcCNNF9OYErUOuQ8Kw4_LmSUjfFZYtB5089K3/s1600/image4.png"/></a></div>
<p>
Drilling into any of the folders can show the stats per file:
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOriiAed0ZNESYwW9RmcuU2oVeP83uOgquDQIUlnCOf-beY-ptM_5Fzdd7SMnSs_Frn2FUiUd1SOHoPpOe3nWE70u43gY_2jgw7XLbvk0XbQbl9Aucly1otrX4gSZalNoHFm7Ek1ed_HxNOWfssiUzaq9hoy46pZ_9bbSbZi3gI2-QchJp8ZmRUoXdnvIt/s1600/image2.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="224" data-original-width="856" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOriiAed0ZNESYwW9RmcuU2oVeP83uOgquDQIUlnCOf-beY-ptM_5Fzdd7SMnSs_Frn2FUiUd1SOHoPpOe3nWE70u43gY_2jgw7XLbvk0XbQbl9Aucly1otrX4gSZalNoHFm7Ek1ed_HxNOWfssiUzaq9hoy46pZ_9bbSbZi3gI2-QchJp8ZmRUoXdnvIt/s1600/image2.png"/></a></div>
<p>
Further clicking on one of the files shows the lines that were touched and lines that never got coverage. In the example below, the first line has been fuzzed ~5 million times, but the fuzzer never makes it into lines 3 and 4, indicating a gap in the coverage for this fuzzer.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsf2OY4HnTtayyK7r0Fd_yZ043UALUgyDwHhQxlk1TSju_VIALC-q5YV6N9DMMM_7hiJ1Pg8raiwG-8av3f9I0QMQ8_LWIbvZaavNC5n7ARb6p1Xdw12ajvaKX5kk4gl-rxXPHOOwLarFtALrB_K29xGoAIGE0L29N78apLrfFWo5y7VW80kcffdvHU344/s1600/Screenshot%202023-08-29%208.37.16%20AM.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="135" data-original-width="821" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgsf2OY4HnTtayyK7r0Fd_yZ043UALUgyDwHhQxlk1TSju_VIALC-q5YV6N9DMMM_7hiJ1Pg8raiwG-8av3f9I0QMQ8_LWIbvZaavNC5n7ARb6p1Xdw12ajvaKX5kk4gl-rxXPHOOwLarFtALrB_K29xGoAIGE0L29N78apLrfFWo5y7VW80kcffdvHU344/s1600/Screenshot%202023-08-29%208.37.16%20AM.png"/></a></div>
<p>
We have dashboards internally that measure our fuzzing coverage across our entire codebase. In order to generate these coverage dashboards yourself, you follow these <a href="https://source.android.com/docs/security/test/libfuzzer#generate-line-coverage">steps</a>.
</p>
<p>
Another measurement of the quality of the fuzzers is how many fuzzing iterations can be done in one second. It has a direct relationship with the computation power and the complexity of the fuzz target. However, this parameter alone can not measure how good or effective the fuzzing is.
</p>
<h3>How we handle fuzzer bugs</h3>
<p>
Android fuzzing utilizes the <a href="https://google.github.io/clusterfuzz/">Clusterfuzz</a> fuzzing infrastructure to handle any found crashes and file a ticket to the Android security team. Android security makes an assessment of the crash based on the <a href="https://source.android.com/docs/security/overview/updates-resources#severity">Android Severity Guidelines</a> and then routes the vulnerability to the proper team for remediation. This entire process of finding the reproducible crash, routing to Android Security, and then assigning the issue to a team responsible can take as little as two hours, and up to a week depending on the type of crash and the severity of the vulnerability.
</p>
<p>
One example of a recent fuzzer success is (<a href="https://nvd.nist.gov/vuln/detail/CVE-2022-20473">CVE 2022-20473</a>), where an internal team wrote a 20-line fuzzer and submitted it to run on Android fuzzing infra. Within a day, the fuzzer was ingested and pushed to our fuzzing infrastructure to begin fuzzing, and shortly found a critical severity vulnerability! A patch for this CVE has been applied by the service team.
</p>
<p>
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9Wib8xdCBTVH5U6OzcDIffYBrH0zGGUcgkVOdcJ0q4zyhFGR6yTnWxnchXL-36QYtugsfFrjxrHOWOQwOBXTRGPRWKZ7BLjAKjdtQvF5IuCKIf8XX6Jf_CzzWLjlLRa1-SH2FECWeeneV1X2L1rUcx_Mf01zO4VtisrYE6mUYo4tKApjX3bJdOC3Mw2ve/s1600/image8.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="636" data-original-width="1999" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi9Wib8xdCBTVH5U6OzcDIffYBrH0zGGUcgkVOdcJ0q4zyhFGR6yTnWxnchXL-36QYtugsfFrjxrHOWOQwOBXTRGPRWKZ7BLjAKjdtQvF5IuCKIf8XX6Jf_CzzWLjlLRa1-SH2FECWeeneV1X2L1rUcx_Mf01zO4VtisrYE6mUYo4tKApjX3bJdOC3Mw2ve/s1600/image8.png"/></a></div>
<h3>Why Android Continues to Invest in Fuzzing</h3>
<h2>Protection Against Code Regressions</h2>
<p>
The Android Open Source Project (AOSP) is a large and complex project with many contributors. As a result, there are thousands of changes made to the project every day. These changes can be anything from small bug fixes to large feature additions, and fuzzing helps to find vulnerabilities that may be inadvertently introduced and not caught during code review.
</p>
<p>
Continuous fuzzing has helped to find these vulnerabilities before they are introduced in production and exploited by attackers. One real-life example is (<a href="https://nvd.nist.gov/vuln/detail/CVE-2023-21041">CVE-2023-21041</a>), a vulnerability discovered by a fuzzer written three years ago. This vulnerability affected Android firmware and could have led to local escalation of privilege with no additional execution privileges needed. This fuzzer was running for many years with limited findings until a code regression led to the introduction of this vulnerability. This CVE has since been patched.
</p>
<h2>Protection against unsafe memory language pitfalls</h2>
<p>
Android has been a huge proponent of <a href="https://source.android.com/docs/setup/build/rust/building-rust-modules/overview">Rust</a>, with Android 13 being the first Android release with the majority of new code in a <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">memory safe language</a>. The amount of new memory-unsafe code entering Android has decreased, but there are still millions of lines of code that remain, hence the need for fuzzing persists.
</p>
<h2>No <del>One</del> Code is Safe: Fuzzing code in memory-safe languages</h2>
<p>
Our work does not stop with non-memory unsafe languages, and we encourage fuzzer development in languages like Rust as well. While fuzzing won’t find common vulnerabilities that you would expect to see memory unsafe languages like C/C++, there have been numerous non-security issues discovered and remediated which contribute to the overall stability of Android.
</p>
<h3>Fuzzing Challenges</h3>
<p>
In addition to generic C/C++ binaries issues such as missing dependencies, fuzzers can have their own classes of problems:
</p>
<p>
<strong>Low executions per second</strong>: in order to fuzz efficiently, the number of mutations has to be in the order of hundreds per second otherwise the fuzzing will take a very long time to cover the code. We addressed this issue by adding a set of alerts that continuously monitor the health of the fuzzers as well as any sudden drop in coverage. Once a fuzzer is identified as underperforming, an automated email is sent to the fuzzer author with details to help them improve the fuzzer.
</p>
<p>
<strong>Fuzzing the wrong code:</strong> Like all resources, fuzzing resources are limited. We want to ensure that those resources give us the highest return, and that generally means devoting them towards fuzzing code that processes untrusted (i.e. potentially attacker controlled) inputs. This can cover any way that the phone can receive input including Bluetooth, NFC, USB, web, etc. Parsing structured input is particularly interesting since there is room for programming errors due to specs complexity. Code that generates output is not particularly interesting to fuzz. Similarly internal code that is not exposed publicly is also less of a security concern. We addressed this issue by identifying the most vulnerable code (see the following section).
</p>
<h3>What to fuzz </h3>
<p>
In order to fuzz the most important components of the Android source code, we focus on libraries that have:
</p>
<ol>
<li>A history of vulnerabilities: the history should not be the distant history since context change but more focus on the last 12 months.
<li>Recent code changes: research indicates that more vulnerabilities are found in recently changed code than code that is more stable.
<li>Remote access: vulnerabilities in code that are reachable remotely can be critical.
<li>Privileged: Similarly to #3, vulnerabilities in code that runs in privileged processes can be critical.
</li>
</ol>
<h3>How to submit a fuzzer to AOSP </h3>
<p>
We’re constantly writing and improving fuzzers internally to cover some of the most sensitive areas of Android, but there is always room for improvement. If you’d like to get started writing your own fuzzer for an area of AOSP, you’re welcome to do so to make Android more secure (<a href="https://android-review.googlesource.com/c/platform/system/keymaster/+/1790088">example CL</a>):
</p>
<ol>
<li>Get <a href="https://source.android.com/docs/security/test/libfuzzer">Android source code</a>
<li>Have a testing phone?
<ul>
<li>Yes!
<ul>
<li><a href="https://source.android.com/docs/security/test/libfuzzer#build-example">Flash it and connect to it.</a>
</li>
</ul>
<li>No? Don’t worry.
<ul>
<li>Install <a href="https://source.android.com/docs/setup/create/cuttlefish-use">cuttlefish</a>: GCE hostable device emulator
</ul>
</ul>
<li>Write a <a href="https://source.android.com/docs/security/test/libfuzzer#write-a-fuzzer">fuzz target</a> (follow guidelines in ‘What to fuzz’ section)
<li><a href="https://source.android.com/docs/setup/contribute/submit-patches">Upload</a> your fuzzer to AOSP.
</li>
</ol>
<p>
Get started by reading our documentation on <a href="https://source.android.com/docs/security/test/libfuzzer">Fuzzing with libFuzzer</a> and check your fuzzer into the <a href="https://android-review.googlesource.com/q/status:open+-is:wip">Android Open Source project</a>. If your fuzzer finds a bug, you can submit it to the <a href="https://bughunters.google.com/about/rules/6171833274204160/android-and-google-devices-security-reward-program-rules">Android Bug Bounty Program</a> and could be eligible for a reward!
</p>
Have questions about this post or want to get in touch with our team? We want to hear from you! Please reach out by emailing us directly at <a href = "mailto:android-fuzzing-external@google.com">android-fuzzing-external@google.com</a>.Edward Fernandezhttp://www.blogger.com/profile/03784424747198152685noreply@blogger.com0