Clean Code’ is a simple concept rooted in common sense. This software writing principle cropped up some 50 years ago and might seem quaint in today’s era of speedy software development. Related: Setting IoT security standards At Black Hat

Black Hat Fireside Chat: Why using ‘Clean Code’ is paramount in speedy software development

Global Code Analysis Tool: Scope of the Report: Our comprehensive market analysis delves into the Global Code Analysis Tool Market, providing profound insights into its dimensions, emerging trends ...

Code Analysis Tool Market Forecast 2023-2030: Global Outlook | SonarSource, Kiuwan, Morphis Tech

The report titled Development Analytics Tools Market 2023-2030 has recently been added by Infinity Business Insights to get a stronger and more effective business outlook. This includes ...

Development Analytics Tools Market Size 2023 by Share, Financial Summary, and Growth Opportunity to 2030 | SonarSource, GitPrime, Waydev

Attackers could remotely leverage a critical TeamCity CI/CD server vulnerability to facilitate arbitrary code execution and server takeovers, according to SecurityWeek.

Server hijacking likely with exploitation of critical TeamCity vulnerability

"The SQL injections were exploitable and had the ability to disclose the PostgreSQL server's version and execute arbitrary commands on the system," said SonarSource, which recently identified ...

RCE attacks possible with Gentoo Soko SQL flaws

Critical SQL injection vulnerabilities found in Gentoo Soko! Exploiting these flaws could lead to remote code execution (RCE) on affected systems.

Critical SQL Injection Flaws Expose Gentoo Soko to Remote Code Execution

The MarketWatch News Department was not involved in the creation of this content. Jun 16, 2023 (Heraldkeepers) -- The Static Code Analysis Software Market is an in-depth marketing research that examines primary and secondary market drivers, market share ...

Static Code Analysis Software Market Overview 2023 to 2030, Future Trends and Forecast | SonarSource S.A, Coverity, JetBrains

And the good folks at SonarSource found a nasty, potential supply chain attack in this tool, when used in the Packagist repository. The problem is the support for arbitrary README filenames.

Wireless Hacking

With this investment, Permira is the third largest shareholder in the company, following Bain Capital and Maverick Capital.

Permira to invest in cybersecurity firm BioCatch

Reviewpad, the platform that helps developers by automating the code review process, has raised a €1M pre-seed round.

Artificial Intelligence: Reviewpad Raises €1M to Streamline Code Reviews by Leveraging AI Tech

SonarSource

sonarsource

We've been working recently on adding rules to help write better regular expressions in Java. I've talked already about rules to and with . Those rules help you make sure your regular expressions are accurate and maintainable. Today I will show you how to make sure that the regular expression won’t crash your program. find errors in character classes the use of boundary markers and overly complex regular expressions As a brief reminder: regular expressions are a terse and powerful mechanism for matching patterns in strings. Part of the power of regular expressions is that a pattern can concisely describe a string that might be much larger than the original pattern. Sometimes much, much larger. Sometimes enough to overflow the stack and crash your application. Due to the way regular expression matching is implemented in Java (and many other languages/libraries), matching a pattern may - depending on the regex - require stack space proportional to the length of the input. This means large inputs could cause the program to crash with a when you try to use the regex. StackOverflowException We ran into this problem in our own code when analysis tried  to find comments containing at least one word with more than a given number of characters. For this we used a regex similar to , which would cause the analysis to crash on source files containing sufficiently long comments. To prevent problems like this for our future selves as well as for you, dear users, we implemented a rule which can detect problems like this: #(.|\n)*\w{3,} java:S5998 - Regular expressions should not overflow the stack In theory any regex is susceptible to stack overflow if it contains a repetition that contains some sort of branching, such as another repetition or an alternative. However, for some regular expressions the length of input required to make them crash is much larger than for others. To account for this, our rule tries to estimate how much stack space a regular expression will consume relative to the input size and only raises issues on regular expressions whose stack consumption exceeds a configurable threshold. One way to avoid stack consumption in regular expressions is to use . Possessive quantifiers are created by adding a to a quantifier (e.g. instead of ). Doing so disables . In addition to avoiding issues with catastrophic backtracking, making a quantifier possessive allows a pattern to be matched without consuming stack space. The problem is that sometimes backtracking is necessary and using possessive quantifiers to disable it may leave you with a regex that can never match any input. possessive quantifiers + x*+ x* backtracking Consider for example the regex I cited above: : If we try to fix the stack overflow by making the quantifier possessive, we end up with . This regex can never match anything because any input that could be matched by will already have been matched by and, being possessive, it won’t give it back. Luckily we have a rule that warns you about issues like this: #(.|\n)*\w{3,} #(.|\n)*+\w{3,} \w (.|\n)*+ java:S5994 - Regex patterns following a possessive quantifier should not always fail So how would one properly fix this issue? In our case, we decided to get rid of the regex altogether since there wasn’t really a good reason to use a regex here. But the regex could also be salvaged easily enough by getting rid of the alternation. The intent of was to match any character, including line breaks, because . does not match line breaks by default. The proper way to match any character would be to enable the flag, which can also be enabled from within the regex using . So either or would work (the latter of which also works when using methods that don’t involve ). .|\n DOTALL (?s) Pattern.compile(“#.*\\w{3,}”, Pattern.DOTALL) ”(?s)#.*\\{3,}” Pattern.compile Another regex susceptible to stack overflows would be something as simple as . Here the alternation can be removed by using a character class instead, which doesn’t require stack space to match: . Luckily we have a rule that finds alternations that can be replaced by character classes and suggests just that replacement like in the following code: (a|b)* [ab]* Spring Boot java:S6035 - Single-character alternations in regular expressions should be replaced with character classes Regular expressions are a truly powerful feature. Because great power comes with great responsibility (and great opportunities for screwing things up, or confusing your colleagues and future-you), we now offer a total of 24 rules targeting regular expressions. They are all available today in SonarCloud. These three rules around avoiding stack overflows are available starting from SonarQube 8.7, but the rest are already released. You can see them all at . rules.sonarsource.com Finally, note that some of the rules perform in-depth automata-based analyses on your regular expressions to identify issues as accurately as possible. This is an extremely promising approach and we will probably discuss this feature in a future blog post because we truly believe it’s amazing. This will for sure bring A LOT to regex code quality in general, so stay tuned!

Avoid Stack Overflows By Writing Regexes Properly

We've been working recently on writing rules for detecting errors in regular expressions in Java code. Regular expressions are a common feature of modern programming languages, and it's easy to take them for granted. By their very nature, they pack a lot of power into terse little packages and unfortunately that introduces a lot of room for error. In I talked about the kinds of errors our newly implemented rule about character classes found in open source code. my last post Today, I’ll talk about boundaries, another regex feature that can lead to bugs when used incorrectly, and a rule of ours that can help you avoid such issues. I’ll also talk about complexity and maintainability in regular expressions and our rule that can help you find regular expressions that are too complex. Boundaries Boundary markers such as and allow you to anchor the regex pattern to the beginning and end of the line (or string depending on which flags you use) respectively. This means that when you want to match a literal or , you need to escape these special characters with a backslash. ^ $ ^ $ And if you fail to escape or then you may end up with a pattern that doesn't match anything at all. In order to detect such problems, we offer a rule ( ) pointing out cases where a boundary is used in a way such that it can never produce a successful match. Here’s one example of this problem that we found while checking code on GitHub: ^ $ java:S5996 - Regex boundaries should not be used in a way that can never be matched Pattern.compile( ) "^[a-zA-Z][a-zA-Z0-9_.][@.](!#$%&*()-+=^){8,30}$" Source (UserPassword.java) Another case where this rule applies is if you use an end-of-line/string boundary at the beginning or a beginning-of-line/string at the end. This could be a case of confusing the meaning of and : ^ $ Pattern.compile( ) ".*A^" Source (MongodbDocumentSerializerTest.java) Complexity Regular expressions are powerful and often terse. Unfortunately, that terseness often makes them hard to read and understand for your teammates, for readers of your code, and even for your future self. Ultimately, they often become a maintainability nightmare. Consider for example the following regex for matching dates: "^(?:(?:31(\\/|-|\\.)(?:0?[13578]|1[02]))\\1|(?:(?:29|30)(\\/|-|\\.)(?:0?[13-9]|1[0-2])\\2))(?:(?:1[6-9]|[2-9]\\d)?\\d{2})$|^(?:29(\\/|-|\\.)0?2\\3(?:(?:(?:1[6-9]|[2-9]\\d)?(?:0[48]|[2468][048]|[13579][26])|(?:(?:16|[2468][048]|[3579][26])00))))$|^(?:0?[1-9]|1\\d|2[0-8])(\\/|-|\\.)(?:(?:0?[1-9])|(?:1[0-2]))\\4(?:(?:1[6-9]|[2-9]\\d)?\\d{2})$" Can you tell what kinds of dates would be matched by this? Would you easily be able to add additional types of dates to it or remove types? Me neither. Another example, one might say the example, of a complicated regex is one that is commonly used to match email addresses, which can be found (and you’ll see lots of versions of it flying around on the internet). I won’t include it in this blog post for space reasons, but it’s more than 6,000(!) characters long. here And perhaps the worst part is that we’ve been guilty of using this regex internally. To us, reusing an overly complicated regex without understanding it sounds like a trap. Whether it works or not may depend on your situation, including which email addresses you want to consider valid and which invalid. For example, if you wanted to write an email to yourself at your local mail server, the mail application shouldn't stop you from addressing the mail to `host@localhost` (a.k.a. Local Host), but when validating email addresses for a web form, you might want to restrict addresses to non-local domains. Now there probably aren’t many people who would be able to tell whether the "standard" email regex would accept `host@localhost` or not by just looking at the code. And certainly it would be a decidedly non-trivial engineering effort to change it to not accept an `@localhost` address if it does (or vice-versa). To help you keep track of complicated regular expressions you’re using, SonarQube, SonarCloud and SonarLint offer a dedicated rule for finding regular expressions with complexity that exceeds a configurable threshold: . java:S5843 - Regular expressions should not be too complicated This rule is inspired by the concept, developed by SonarSource, and takes into account how all the regex operators, combined with each other, raise the complexity of a given regex. Here’s what it says about the date regex from above: cognitive complexity Now, what makes this regex so complicated? The fact that it verifies that the date doesn’t just fit the general format of a date, but is a valid date to the point that it only allows the 29th of February if the year is a leap year. In this case, it’d be preferable to use a simpler regex to only match the general format and then write a plain Java method to check that the numbers make sense and whether it’s a leap year. An even better solution would be to delegate that logic to a dedicated library and use it from there. When you find yourself in the process of writing (or reusing) such a complicated regex, you should first ask yourself whether you couldn’t easily solve the problem without using regular expressions at all. The next best option is using a combination of multiple simpler regular expressions applied in chain, one after another (such as first using one regex to split the input string and then using others to process each part of the string). If that’s not possible, try to split the regex into multiple parts and document each part or at least assign it to a variable with a meaningful name. Then, once you’ve gotten your regular expression(s) down to a manageable size, the rules for specific features like character classes and boundary markers can help you make sure your pattern matches what you think it does. At that point, with complexity and accuracy handled, you're nearly done. All that's left to worry about is stability - as in the stability of your application. That's right, a regex gone bad can bring down your application. So next time, I'll talk about the humble regex's potential for stack overflows, and what we've done to help you prevent it.

Getting Regex Boundaries Right: A How-To Guide

Regular expressions are a concise and powerful tool for processing text. However, they also come with a steep learning curve and plenty of opportunities to make mistakes. This is the first in a series of posts about some specific pitfalls of Java regular expressions that can lead to bugs, code that’s hard to understand, or worse: code that could crash your application. In this series we will give you some examples of issues in real code caused by these pitfalls, and discuss strategies (and rules!) for writing better, more readable and maintainable regular expressions. In this post I’ll start with pitfalls related to a very common feature of regular expressions: character classes. Note that writing this blog post has been made possible thanks to the group effort of the whole SonarSource Java analysis team. Transforming our initial ideas into such features is a great collective achievement, which I’ll now share with you, speaking for the team! Character classes allow the regex engine to match only one out of several characters. For instance: The character class can match either an or a .You can also use ranges inside character classes: matches any character between and .You can inverse or negate character classes with : By starting the character class with a single you negate everything that follows in the class. So matches anything that's not a lowercase ASCII letter. [xy] x y [e-p] e p ^ ^ [^a-z] Where it starts to be tricky is that some characters have different meanings inside character classes than they do outside. The best example of this is probably the character which gains the special meaning of creating ranges when used inside a character class. hyphen/minus - To match a literal , you can escape it or move the to the beginning or end of the character class. Another example is the multipliers. For instance outside a character class, means "repeated any number of times". Inside a character class it just means "asterisk". - \- - * You probably think this "Character Classes" concept is easy and well understood by developers. However, after running our analyzer on a few GitHub open-source projects, we realized that it might not be the case at all. So let's take a look at real code and see how creative developers can be! Problem 1: Wrong use of separators There is a lot of confusion around the character. Outside of a character class, it is an alternation operator. So it would allow you to select "red" or "blue", like so: . But inside a character class, it's just a normal character with no special behavior. For example in this “mobile-phone number” matcher: | red|blue Pattern.compile( ) "^1[3|4|5|7|8][0-9]{9}$" Source (ValidateUtils.java) The author should replace with in the pattern. [3|4|5|7|8] [34578] Other developers make the same mistake with commas, as in this example: …[ , , , - ]… 0 2 3 5 9 Source (PhoneRecognition.java) And the negation symbol should only be used at the beginning of the character class and not before each element, like in this code: ^ NanoHTTPD …[^/^ ^;^,]… Source (ContentType.java) Problem 2: Wrong character A more subtle potential bug is the uppercase and lowercase mix in character ranges, like in the code: Apache Camel …[\\.|a-z|A-z| - ]… 0 9 Source (KafkaHeaderFilterStrategy.java) Do you see the bug? Not the wrong use, the other one?  Because of the second lower-case , the range matches characters in the ASCII table from to , plus , , , , , , and adds from to on top of that. Isn't it strange? So now it should take you only one second to find a bug in this code which is commented "defined by RFC7230 section 3.2.6" for this expression: | z [A-z] A Z [ \ ] ^ _ ` a z Elasticsearch Pattern.compile( ); "[a-zA-z0-9!#$%&'*+\\-.\\^_`|~]+" Source (RestRequest.java) Unfortunately, RFC7230 does not allow , , in HTTP header field values, so it's definitely a bug. A similar bug could also occur when you want to match the character - and forget to escape it or move it to the first position in the class (where it would lose its special meaning). Can you spot which - character is wrong in the following code? [ \ ] Jenkins USERINFO_CHARS_REGEX = ; "[a-zA-Z0-9%-._~!$&'()*+,;=]" Source (UrlValidator.java) It's the one in the range ; it does not match 3 characters but and because the matched characters are also present after in the character class, we know that the range was not intentional. Luckily, this expression will only fail to match the character , but sometimes this confusion can have a bigger impact: %-. %&'()*+,-. %-. - String safetextRegex = ; "^[a-zA-Z0-9 .,;-_€@$äÄöÖüÜ!?#&=]+$" Source (ValidationBean.java) Nice variable name, but unfortunately this character class is most probably not as safe as expected by its initial writer. Indeed, here does not match 3 characters, but 37! ;-_ And don't forget that a range can only match one and only one character. If you want to match characters '0' '1' '2' '3', you can use .  But what do you think the following code is supposed to match? [0-3] Apache Hadoop Pattern.compile( ) "acl[0-31]" Source (AoclDiagnosticOutputParser.java) could just be a redundancy and not a bug. But, if the intention was to match an acl number as defined by Intel from to , then it's a bug. Likewise, matching uppercase and lowercase requires two character ranges and not only one like in this code: 1 acl0 acl31 [A-Za-z] Apache Geode …[aA-zZ0- -_.]… 9 Source (CreateLuceneCommandParametersValidator.java) Problem 3: Wrong regex operator Sometimes alternations like are wrongly written using character classes. Can you spot the bug in the following source code? (jpg|png|gif) Alibaba's Tangram Pattern.compile( ); "(\\d+)x(\\d+)(_?q\\d+)?(\\.[jpg|png|gif])" Source (Utils.java) Good to know, and are just normal characters when used in character classes and lose their meaning as quantifiers. So in this next example, why would you add a ? inside a character class? * ? String VALUE = ; "[[^\"]?]+" // anything but a " in "" Source (HiveHistoryUtil.java) It's a complicated way to write , and probably the intention was actually to write . [^\"]+ [^\"]* The above bugs were found by our new rule . The initial goal of this rule was to spot tiny misunderstandings like: java:S5869 - Character classes in regular expressions should not contain the same character twice But in the end, the findings far exceeded our expectations and will ultimately prevent some very painful bugs in your applications. S5869 is available today in SonarQube, SonarCloud and SonarLint. It was Voltaire who first said that with great power comes great responsibility. But what we've learned in implementing rules for regular expressions is that with the great power of regular expressions, also come great challenges to write them well. In this post I talked about what we found with rule S5869, but it's only one of the regex rules we've been working on. Next time I'll talk about regex boundaries and complexity. Previously published at https://blog.sonarsource.com/regular-expressions-present-challenges

Regexes or Regular Expressions And The Common Mistakes Programmers Make While Using Them

At SonarSource, we believe that just like developer-led Code Quality did a decade ago. Finding Vulnerabilities immediately after they're introduced instead of weeks later just makes more sense. But what does that mean for the risk and compliance teams that lead those efforts today? Those jobs don't go away. Actually, they get more interesting and possibly even more important. developer-led Code Security will inevitably come to dominate the SAST market Today, security auditors really aren't in an enviable spot. First they're working with tools that take a no-news-is-good-news approach; tools that raise issues for anything even remotely suspicious. So Security auditors spend a significant chunk of their time on the drudge work of identifying the few things that really need action in a sea of false positives. Once they've done that, they can't actually fix the code themselves. Instead, they have to deliver the bad news to the folks who actually can - the developers - and convince them to switch focus from delivering business value to correcting code they haven't thought about in weeks. stale It doesn't sound like a dream job, does it? But with developer-led Code Security, things get better. First, . We put those Vulnerabilities front-and-center for developers, so they're handled while the code is still fresh in mind. if we raise a Vulnerability, you there's something to fix know And because it's integrated into their workflow, developers will fix most Vulnerabilities and review Security Hotspots as a matter of course. That lets security auditors move out of the enforcer role. Instead of "You must fix this old code!", they can shift into collaboration and oversight. Security Hotspots The first opportunity here is around . We provide a specialized interface and background data for developer review of Security Hotspots. But even so, some Security Hotspots won't be straightforward. This is where security auditors can step in and guide developers through the tricky cases to reach a good resolution. Security Hotspots And that leads to the oversight role auditors can now assume in the process. They can begin to look at things like: what rules are in the Quality Profiles and which conditions are in the Quality Gates? And then they can go deeper: are there patterns in the types of issues being raised that additional training can help with? Pattern Spotting with SonarQube That pattern-spotting can be done manually on the issues page, but offers reports that can help identify patterns, and more importantly help translate them to the development team in language coders are familiar with. Either way, under the new paradigm, security auditors spend less time identifying the real issues and more time making sure they get properly addressed. SonarQube's Enterprise Edition With the shift to developer-led Code Security, risk and compliance folks move out of the middle-man position. They're no longer the bearers of bad news. Instead, they become collaborators with development teams. Together, they can ensure that software is more secure. And that's good for all of us. Previously published at https://blog.sonarsource.com/security-auditors-the-cinderella-story-of-sast

Developer-Led Code Security Will Dominate the SAST Market

I've said before that accurate analysis and the killing of the noise are the keys to developer-led security. As an issue of credibility, developers have to that when we raise a Vulnerability issue, there something to fix. know is But there's another class of security issues, a class that can never be clear-cut. Each issue in this class has a 50/50 chance of being a real Vulnerability or of being no big deal at all. At SonarSource, our SAST mission is to eliminate false positives but we can't simply ignore this class because these issues represent real vulnerabilities and our goal is to provide a complete SAST offering. can Of course, we never want to simply dump everything on developers and leave them to sort it out. So we've segregated these issues into what we call Security Hotspots. And we see this separation as key to retaining credibility and keeping developers engaged in the SAST process. I've about what kinds of things fall into the Security Hotspot class. One good example is the use of a weak hashing algorithm (i.e. one that's too easy to reverse). It's easy for SAST analysis to spot the use of a weak algorithm. What's harder is knowing if the value being hashed ought to be better-protected. For instance, if the value I'm hashing contains your credit card number, CVV, and expiration date… well it's obviously a problem to use a weak algorithm. But if instead that value contains your favorite milkshake flavor (double peanut butter, please!) then there's no issue. Knowing the difference? Well, that takes a human. talked before Once we get that human involved - ideally the developer who wrote the code - we want to make sure there's no perception that we're crying wolf. So we've built an entirely separate interface for reviewing Security Hotspots. That does a couple things. First, it takes you out of the context of Vulnerabilities and takes the sting out having something security-related raised on your code. Instead of presenting it as "There's something broken and you have to fix it!" it's more like "Hey, can you take a look at this?" After all, it's not what you say, it's how you say it, right? The other thing having a separate interface allows us to do is present the Security Hotspot in the context of the background information developers need to assess whether or not there's actually a problem, and what the consequences could be. If there's not, it's easy to dismiss the Security Hotspot you're on and move automatically to the next one. If there is a problem, or if you're not sure, then you can assign it for fixing or further review. Security Hotspots are a new concept in the SAST world, and we believe they're an important part of shifting the overall SAST paradigm. We know that maintaining engagement is key to making developer-led code security work, and our strategy for this is two-pronged: minimize False Positives among true Vulnerabilities, and properly label and route Security Hotspots. It would be great if SAST analysis could tell the difference between credit card info and milkshake flavors so we could eliminate the Security Hotspot category altogether. But just like sports needs referees, Security Hotspots need developers to make the call. At least as a developer, you get to referee your own code. Previously published at https://blog.sonarsource.com/security-hotspots-maintain-engagement-in-developer-led-security

Developer-led Security: Hotspots Continue To Maintain Engagement

Recently I talked about how SonarSource is from targeting security compliance auditors to developer-led code security. Today I'd like to talk about the most fundamental part of that change: accuracy in the issues we raise. shifting the SAST paradigm Most SAST tools target security compliance auditors. Their goal is to raise an issue for anything even remotely suspicious. There's no fear of false positives for those tools because the auditors will figure it out; after all it's the auditors' to sort the wheat from the chaff and the signal from the noise. job But for years now the rallying cry at SonarSource has been "Kill the noise!" As a developer-first company, we know there's little tolerance among developers for crying wolf. So our guiding principle has been to prefer "reasonable" false negatives to raising false positives. What does that mean in practical terms? Well, let's play with some numbers. Let's say you have a codebase with 12 Vulnerabilities. That's 12 things that absolutely need fixing. A typical SAST analysis might raise 500 issues in total, and then the auditors will spend x weeks sorting through that to bring you, the developer, the audit result maybe a month or so after you've moved on to other code. Not an appealing scenario, is it? So okay, let's eliminate the lag time by pivoting to developer-led code security. Now, instead of taking weeks to sort through the SAST report, the auditors dump it on your desk. They expect you - as the developer - to find and fix the true Vulnerabilities. This scenario's even worse, both for you and for the security of the codebase. Because let's be honest, it won't take many false positives for you to throw up your hands and declare the whole thing a waste of time. Now, gets fixed. nothing At SonarSource, we're keenly aware of that. That's why we accept reasonable false negatives. Instead of raising 12 real Vulnerabilities that are ultimately lost and ignored in a sea of false positives, we'd rather raise only 10 real Vulnerabilities that and miss the other two. actually get fixed Don't misunderstand. We're not missing those other two (theoretical!) issues because we're sloppy or lazy. Sometimes in implementing a rule you have to strike a balance between catching every single issue … and also getting a few False Positives in the net, or tuning the rule sensitivity down to eliminate False Positives… and missing a few real issues at the same time. SonarSource developer Loic Joly recently gave on striking that delicate balance. As he explained, when we're faced with this choice, we're going to choose false every time. a rule implementer's perspective negatives It's an issue of credibility. As I said earlier, we know that developers don't have patience with False Positives. So we make sure that when we raise an issue, there's something to fix. That doesn't mean we never raise False Positives. We're human too, and if developers were perfect, you wouldn't need us to start with. But our mission is giving you an accurate SAST analysis, and killing the noise. And it makes all the difference. Previously published at https://blog.sonarsource.com/blazing-a-trail-on-the-sast-road-less-traveled-by

Why we Prefer "Reasonable" False Negatives to Raising False Positives 

In 2008 SonarSource upended the static analysis market for code quality and reliability. Today it's doing it again for code security. When SonarSource was founded in 2008, code quality was the realm of specialists, who typically reported their results just before production deployment, weeks or months after the code was written. Inevitably, by then the authors had moved on to other things - mentally if not physically - and getting anything fixed was a contest of wills. Fast-forward to today, and you find that most teams have integrated static analysis from the first keystroke, starting with SonarLint in the IDE and moving forward with SonarQube and SonarCloud in the CI/CD workflow. With code quality and reliability integrated into the development workflow, the bad-old-days of specialist-owned, quarterly analysis may seem as quaintly horrific as an old film. But in the realm of security analysis, much like , that paradigm has so far refused to die. Today most SAST (Static Application Security Testing) tools are still owned and run outside the development team. Results are delivered intermittently, and if the past is a guide, no one really code security. The security team can't own it because they can't it; they can't change the code. And developers can't own it because they can't of it. They don't own the rules, the timelines, or the reports, so they own the result. Boris Karloff Karloff's Mummy owns impact take ownership can't Fortunately, you have the tools in hand today to break that pattern and shift code security - like code quality and reliability - to developers. With the SonarSource model: SAST is integrated into the development workflow by making it part of the tools developers already use: SonarQube and SonarCloud analysis.That means quick security feedback, and - because the code is still fresh in mind - efficient, effective fixes.Giving developers control of SAST tooling lets them take ownership of code security, so pushback turns into pride of workmanship. Those are the changes that are visible on the surface. But making this work - making sure developers accept, respect, and act on SAST feedback - means a more fundamental shift. It's not enough to change how, and when issues are raised. issues are raised has to change too. Most SAST tools raise issues for everything even remotely suspicious and let the auditors sort it out. But developer-led security won't work that way. Developers simply don't have the time or the patience for false positives. Which At SonarSource, we know all about killing False Positives. It's been our mission for years now, first with Bugs and Code Smells and now with Vulnerabilities too. In fact, earlier this year we acquired RIPS Technologies for access to their SAST engine. Our own SAST analysis was already pretty good, but with the addition of RIPS, it's even more precise. But it's not just about technology; the underlying philosophy has to change too. I'll write more on that next time and leave you with this: Good developers to write good (and secure!) code. Let them know they already have the tools to do that - and the - and all you have to do next is just get out of the way. want SonarCloud latest versions of SonarQube

SAST Analysis: No Angst Edition

Hey SonarQube and SonarCloud users! You now have a tool to own Code Security! SonarSource has been hard at work for the last year to give you the tooling to review and improve your code security. We're glad to say that today you have at your fingertips unmatched precision and performance in SAST (Static Application Security Testing) analysis for five languages and counting. We've been working to bring you Code Security for a couple years now, and those efforts took a giant leap forward this spring when we acquired RIPS Technologies. RIPS initially caught our eye with the depth and precision of their PHP analysis. Seeing that, we knew their depth plus our breadth - analysis of 27 languages - would make a great combination. We joined forces in May and immediately started work to combine our two approaches. Since then, we've re-engineered our detection of injection vulnerabilities from the ground up to incorporate the best of RIPS' approach and ours. The result: today you have access to unparalleled precision in security analysis of Java, C#, PHP, Python and JavaScript code, with more languages to come. I can say "unparalleled precision" because we've focused in this work on eliminating false positives. Old-school SAST tools aren't built for developers. They cast a very broad net, raising an issue for everything even remotely suspicious, and make an auditor sort it out. At SonarSource, we know developers don't have time for that. So we've made sure that when we raise an issue, you can be confident there's something to fix. At the same time, we haven't sacrificed performance; analysis is still extremely fast. The best part is that you don't have to learn a new tool. These SAST advancements are part of what you already know and love: SonarCloud and SonarQube. Vulnerabilities and Security Hotspots start in SonarQube Community Edition and injection Vulnerabilities (taint analysis) is available in commercial editions. And they're available today. Getting started is as easy as making sure your Quality Profile (and your version!) is up to date. By adding SAST to SonarCloud and SonarQube, we've put the power to own Code Security in your hands. That represents a fundamental shift, and there's a lot more to say on the topic. I'll save that for the New Year. For now, enjoy the holidays and your new toy: SAST analysis built for developers.

Review and Improve Your Code Security Analysis in SAST

As developers, there always comes a time when we find a bug in production and wonder how it passed all our quality checks. The truth is that we can never be sure our code is bug free. We can only choose the tools and workflows which will find the most bugs without slowing us down too much. SonarQube, SonarLint and SonarCloud are such tools. We used SonarCloud during our recent bug report campaign, which focused on popular projects such as , , , and . The campaign result was quite interesting, since it shows the kind of bugs we can find in a Python project even when its development workflow includes every best practice: code reviews, high test coverage, and the use of one or more linters (flake8, pylint, ...). tensorflow numpy salt sentry biopython Let's go over a few Bugs we found with SonarCloud and see why it is able to detect them when popular linters don't . Reference to an undefined variable SonarCloud can detect buggy references to undefined variables when the variables are defined in another branch. It uses a to deduce that the definition of the variable will never occur before the buggy reference. if-else Control Flow Graph Unreachable code Detecting dead code is easy when it's just after a or a statement. It's a little harder when the is conditional. We use a control flow graph to detect cases where multiple branches exit just before reaching a statement. return raise return Wrong fields in formatted strings It is quite common to reference the wrong field name or index during string formatting. Pylint and Flake8 have rules detecting this problem with string literals, but they miss bugs when the format string is in a variable. Type errors SonarCloud has a type inference engine, which enables it to detect advanced type errors. It uses every bit of information it can find to deduce variable type, including stubs, assignments, and your type annotations.. At the same time, it won't complain if you don't use type annotations, and it's designed to avoid False Positives. Typeshed In this example, control flow analysis is what allows it to understand that is a tuple because it is assigned when is a . The algorithm is able to ignore the later assignments to . state_shape output_shape[1:] output_shape tuple list output_shape Now let's look at some more specific examples. Wrong argument type SonarCloud uses stubs to know the types expected by builtins functions. So here it raises an issue because you get a if you call the builtin on an integer. Typeshed TypeError len Comparisons that don't make sense SonarCloud has many rules detecting code which doesn't make sense. Comparing incompatible types with will never fail, but it will always return False, or True if you use . Here we can see an issue because returns a tuple. == != platform.architecture() Return values from functions without side effects should not be ignored Some function calls have no side effect, i.e. they won't change anything by themselves and their only purpose is to return a value. Thus  there is always a bug when their result is not used. SonarCloud knows an extensive list of such functions. In this example the two strings are not concatenated; the method is called on the second string and the result is discarded, so the value of is "Make sure that your dataset can generate at least ". format warning_msg Unraised exceptions When we review code we usually look at classes, variables and other meaningful symbols and we forget to check little details, such as "is there a raise keyword before my exception". SonarCloud analyzes your whole project to extract type hierarchies. Thus it detects when custom exceptions are discarded, not just the builtin ones. Flake8 is great but not enough Some of the things [SonarCloud] spots are impressive (probably driven by some introspection and/or type inference), not just the simple pattern matching that I am used to in most of the flake8 ecosystem. - Peter J. A. Cock - maintainer of BioPython ( original post here ) This is one of the nice pieces of feedback we received during our bug report campaign. ( !). There's more All the projects we examined use one or more linters, such as Flake8, which is very popular, and is often included in CI workflows. There are very good reasons for Flake8's broad use: it focuses on uncontroversial rules that generate few false positives It checks pep8 style It is fast , and have the same philosophy about speed and false positives. All three target developers, which means that . In addition, SonarCloud and SonarQube can both import Flake8 issues. But most importantly: SonarLint SonarCloud SonarQube we work hard to keep "noise" to a minimum they detect a broader range of issues. Not just style and pattern matching, but a full range of . bugs, code smells and vulnerabilities they help you focus on achieving high quality in recent changes (i.e. ) rather than distracting you with small flaws in old code Clean as You Code they support all the languages in your project. For example if you've got JavaScript or TypeScript alongside your Python, it will be analyzed simultaneously, with no more setup or infrastructure. You can use SonarCloud for free on any open source project and get started with just a few clicks. is also free for unlimited on-premises use. Don't hesitate to share your feedback, good or bad, . It helps us improve our tools everyday. SonarQube Community Edition in our community forum Previously published at https://blog.sonarsource.com/sonarcloud-finds-bugs-in-high-quality-python-projects

The Ultimate Guide To Hassle Free Bugs Fixing In Premium Python Projects By SonarCloud 

Author Robert Collier said that "Success is the sum of small efforts repeated day in and day out." That's especially true when it comes to security. By now we all understand that securing your systems isn't as simple as installing a firewall and calling it a day. Instead, it's multiple actions and strategies in concert, implemented consistently over time. And believe it or not, one small but important strategy is simply writing code that's reliable (bug-free) and maintainable (easy to understand). Yes, I know that sounds too simple, and possibly even self-serving. So in this post I'll lay out some of the evidence for how writing code means you're inherently writing more code. reliable and maintainable secure Poor maintainability contributed to Heartbleed To make the case for how maintainable code contributes to security, I'll start with the Heartbleed Bug. Remember that one? It was a serious vulnerability in OpenSSL that allowed attackers to steal sensitive information with a really trivial attack that . David A. Wheeler teaches a graduate course in secure development at George Mason University. He wrote an of the vulnerability. In it, he laid part of the blame on the difficulty of simply understanding the code involved: "Many of the static techniques for countering Heartbleed-like defects, including manual review, were thwarted because the OpenSSL code is just too complex. Code that is security-sensitive needs to be 'as simple as possible'." XKCD illustrates beautifully extensive analysis When the Heartbleed Bug was eventually found, it was actually detected by human review rather than static analysis. It's worth noting explicitly here that the problem wasn't caught in peer review, but long after the merge by independent security researchers. In his analysis, Wheeler discusses why Heartbleed wasn't found sooner. "Little things like code formatting matter," he says, "since badly-formatted code is much harder for humans to review." Code Smell / Maintainability rules for things like code formatting and naming conventions are often dismissed as trivial, maybe because they're about things so foundational that people take them for granted. As Wheeler points out, that doesn't mean they're not important. Wheeler suggests that attention to maintainability leads to more secure software, and continues that "The goal should be code that is , as opposed to code that is so complicated that ." And of course, that's what Code Smell rules help you do - write code that's maintainable and easy to read so that it's for it to be " obviously right I can’t see any problems possible obviously right". Experts: Bugs and Code Smells are security 'weaknesses' Of course, Wheeler's just one person, and opinions are like belly buttons, right? So let's look at another source: the CWE, which makes the case for both maintainability reliability as contributors to security. and I want to start with the Most Dangerous Software Weaknesses, which is an expert-sourced subset of the CWE. But first, some background: CWE stands for . It's a crowd- (of experts) sourced list of common software and hardware weaknesses that have "security ramifications". It has about 1,300 entries, including quite a few that are used for categorization. The Top 25 is a list of "the most common and impactful issues experienced over the previous two calendar years." Given this build-up it would be reasonable to assume that all 25 CWEs in the list describe security vulnerabilities. But by my count, nearly a third are bugs. Bugs that could to security breaches, but bugs nonetheless. For instance, lucky number 13 in the list is , NULL Pointer Dereference. 2020 CWE Top 25 Common Weakness Enumeration lead CWE-76 In fact by one count, about 60% of CWEs aren't vulnerabilities at all. is the  Software Development view. It "organizes weaknesses around concepts that are frequently used or encountered in software development". It contains 40 sub-categories, including , and . Of the 59 leaf listings under Bad Coding Practices, the first is the beautifully emblematic , Missing Default Case in Switch Statement. CWE-699 Complexity Issues Numeric Errors Bad Coding Practices CWE-478 This is not a rule most people see as important for Code Security. At SonarSource, we don't even class it as a Bug, but as a / Maintainability problem. But its inclusion in the CWE says that experts in the field see it as important for security. Because the small consistent efforts like providing `default` clauses help you write "code that is ". Code Smell obviously right Maintainability + reliability -> secure code With the Heartbleed Bug we saw that maintainability matters for secure code. If you can't understand the code, then you can't be sure it isn't hiding vulnerabilities. Add the CWE into consideration, and you see that experts in the field agree that both maintainability and reliability are important contributors to code security. To be clear, static analysis didn't find the Heartbleed Bug. And there will always be bugs and vulnerabilities that static analysis can't find. For instance, there's no way to know statically that you shouldn't allow a negative integer as a shopping cart quantity. That's the job of peer review. In the case of Heartbleed, peer review failed because the code wasn't understandable / maintainable. Static analysis didn't find Heartbleed, but it could have put the peer reviewers in a better position to spot it on their own. Static analysis tools such as and can easily find maintainability and reliability issues, as well as lots of valuable security issues such as injection vulnerabilities, XSS, XXE, deserialization flaws and more. When all three categories are implemented consistently over time, they'll help you gain control of your code's reliability security. SonarQube SonarCloud and With apologies to Robert Collier, perhaps Because maintainability matters. for code security. Security success is the sum of small Maintainability efforts repeated day in and day out. Especially Previously published at https://blog.sonarsource.com/for-secure-code-maintainability-matters

Stories, News & Mentions about SONARSOURCE

Sonarsource’s Stories on HackerNoon

Sonarsource's News & Mentions