Pinderkent

I read an article today about whether higher-level programming languages like Python, Perl and C# are really that much more productive than a comparatively lower-level language like C. This is not a new line of discussion, by any means. But we're getting to the point where we've been using such higher-level languages for over a decade, and thus have had more of an opportunity to observe and analyze how successfully (or not) they've been used.

In my view, that article comes to the general conclusion that many of the popular claims regarding the benefits of high-level languages versus low-level languages don't hold true. It's suggested that languages like C do indeed have aspects that hamper developer productivity, but high-level languages bring their own, albeit different, set of problems. While a C developer may run into problems with pointers, a Perl programmer might lose a similar amount of time optimizing regular expressions.

I think the conclusions of that article are correct to some extent, but I also think that the greater picture may have been missed. The real impact of languages like Python, Ruby, Perl, JavaScript and PHP isn't that they allowed expert programmers to be marginally more productive. Their greatest "benefit" (or arguably their greatest disadvantage) is that they have allowed average and even poor programmers to accomplish things they couldn't have reasonably done in C.

PHP and JavaScript are good examples of this. As anyone who has used them knows, they are very unremarkable languages. Conceptually and syntactically, they're much like C in many ways, but without some of the aspects of C that average developers often find bothersome. They aim to eliminate manual memory management, for instance. They offer slightly nicer string handling. Their execution environments aren't as tied to the native hardware. But otherwise, the core PHP and JavaScript languages generally offer the same basic functionality that C offers.

By eliminating some of the more difficult aspects of C, even if they're not as flexible or as powerful in many ways, they've made programming accessible to people who otherwise would not have been able to handle it. I've had the misfortune of working with people like this. They can understand the concepts of variables, constants, loops, conditionals, functions and even the basics of OO to some extent. But they're totally unable to understand some of the basic, yet essential, concepts of C. Pointers, for some reason, is a common one. But luckily for such average developers, languages like PHP and JavaScript make their lives easier by getting rid of such constructs and functionality.

So we soon enough see these average developers using languages like PHP and JavaScript to develop applications. In many cases, there's little to nothing preventing the same application from having been developed using C, aside from the inability of the average developer or developers to use C. Anyone who has worked in the industry knows why businesses opt to go with such solutions. Sometimes it is cheaper and easier to hire several PHP and JavaScript developers, instead of just one or two expert C developers. Other times it's because inexperienced or unknowledgeable managers just don't know any better, or have bought into hype and marketing. Regardless, the outcome is typically a system that just barely works, assuming it's not outright broken. Whatever costs might have been saved initially end up becoming far more costly in the long run.

Had those JavaScript and PHP developers been forced to use C, it's likely that we wouldn't have seen any sort of a software system be produced at all. They would've still been struggling with significant memory leaks, segfaults, and sometimes even just getting their code to compile. So we can see the actual main benefit of such higher-level languages; they've reduced the complexity of an otherwise difficult skill down to something that is more palatable to non-experts.

Now, we need to ensure that we don't lump high-level languages like Haskell, Erlang, and Common LISP in with other high-level languages like PHP and JavaScript. They are clearly very different. Haskell, Erlang and Common LISP, for instance, use abstraction to empower the developer. They offer advanced features and techniques that expert developers can build upon to great benefit. This is very different from languages like PHP and JavaScript, which clearly took the C model of computing, and stripped out the parts that make C more awkward for the less-skilled programmers.

Even thought they are significantly higher-level languages than C, languages like Erlang, Haskell and Common LISP haven't become as popular because they still require a high level of knowledge and expertise to use for even the most basic of tasks. So they highlight the important difference between a language being "high-level" and a language being "accessible". Functional languages increase expert programmer productivity with more powerful abstractions; PHP and JavaScript increase average programmer productivity via simplification.

The whole debate with respect to whether high-level languages are better than low-level languages will likely rage for many more years. There are some tasks that just can't be done in languages like JavaScript and PHP, so we will surely see C remain around for a long time. But we likely will see languages like PHP and JavaScript remain around for a similar reason. Unfortunately, that reason won't be about allowing good developers to develop more advanced software more quickly, but rather about letting poor developers continue to put out just barely suitable software systems.

I read an article today suggesting that functional programming hasn't caught on widely because most developers today haven't been convinced that it is necessary. I'm not sure that's completely correct. Those who are truly "in the know" will have been at least investigating functional programming for some time now. The rest can merely be considered stragglers, who don't yet realize that they currently do have, or will have, a need for functional programming in the near future.

Although it was clear several years ago, we can now say with almost complete certainty now that we will soon be seeing consumer-grade PCs with 32 or more logical processors. Intel's Core i7 has recently brought us part of the way there. At the higher end, Sun's UltraSPARC T2, for instance, offers 8 cores for 64 threads of execution. The trend is obviously towards processors offering multiple cores, each of which may offer multiple threads of execution. And today's mainstream programming languages just don't take advantage of such environments too well.

Some industries have dealt with these issues already. That's why Erlang has been around for over 20 years now. Anyone who has worked on telecom systems or even just worked with Erlang knows the benefits that its concurrency model brings. These days, with the advent of consumer-grade multicore CPUs, such benefits are applicable even when using the low-end systems. In fact, to make the most of today's and tomorrow's CPUs, we will need to develop our software to be more concurrent.

To do this effectively, we likely will need to make use of at least some of the techniques pioneered by the functional programming community. Immutability is perhaps the most important and useful technique when writing concurrent software. Strong, static typing as offered by languages like Haskell and Standard ML is also very useful. Well-integrated support for writing concurrent applications, such as that offered by Erlang, is another major benefit.

However, we likely won't see the development community at large using Haskell, ML or Erlang any time soon. What we'll probably see first is a move towards hybrid or multi-paradigm languages. Clojure and Scala are two of the most familiar at this point. Both target the Java platform, which makes them appealing to existing Java users.

Somewhat surprisingly, Microsoft may very well be the ones who bring functional programming to the mainstream, through their F# language that is to be included as part of Visual Studio 2010. Functional programming will quickly be made easily accessible to a large number of application developers, at a critical time when many will likely be looking for ways to better leverage systems with many logical processors.

Unfortunately, Sun seems content with spinning their wheels with Java and worse, JavaFX Script. Given Sun's long history providing back-end hardware and software, especially hardware that offers many threads of execution, it's sort of surprising that they aren't giving more support to languages like Clojure and Scala. These are the sort of languages that give current JVM users the boost they need to remain or even become competitive.

We're starting to see a variety of factors come together now that will make functional programming techniques far more useful to the mainstream software developer. Multicore CPUs are now readily available, including in low-end consumer-grade desktops and laptops. This will start increasing the demand for efficient, scalable and practical methods of making use of these resources. A variety of open source functional languages, like Haskell, Erlang, Scala and Clojure, have shown that they're willing to help meet these demands. Microsoft is going with F#. The need to use functional techniques combined with readily-available language implementations will allow for this flourishing to occur.

I saw an article today talking about a "new wave" of activity within the Perl community. Although things have been looking more positive for Perl, it's still far too early to suggest that there is a Perl resurgence in progress.

The most significant progress the Perl community has made lately is that of actually getting some traction behind Parrot and Perl 6. For most of the past decade, we saw little to nothing substantial. There was a lot of talk, but little in the way of tangible software. What was available was not production-ready by any means, which of course is unsuitable for a pragmatic language like Perl.

Pugs, a Perl 6 implementation written in Haskell, was perhaps the most useful of what was produced during what we might come to refer to as the "Perl Dark Ages" of 2001 to 2008. During 2005, it saw relatively active development, and frequent releases. Unfortunately, it lost momentum during 2006, and since then has essentially stalled. A good indication of this is its very homepage, which at the time of this writing (February 18, 2009) still says "© Copyright 2005-2007, The Pugs Contributors."

The announcement last December suggesting that we'd actually see Parrot 1.0 during the first part of 2009 was a good first step. Indeed, it appears as though the March 2009 release of Parrot 1.0 is still on track. This upcoming release will no doubt be highly publicized online, and will likely raise significant awareness of Parrot and Perl 6. Although it isn't mature yet, this release will be an important milestone in lending some legitimacy to both the Parrot team and to the Perl 6 efforts so far.

A month after the aforementioned announcement regarding Parrot 1.0, we got word that the Rakudo implementation of Perl 6 was going to become more independent of the Parrot project. Over the past month, we have seen this happen to some extent.

Without a production-ready implementation of Perl 6, we can't yet say that there is a true resurgence or "new wave" within the Perl community yet. The recent activity surrounding Parrot and Rakudo has helped put the Perl community on the right track. But we likely won't see any true and widespread excitement arise until we have a decent Perl 6 implementation readily available. Only then can module authors start writing code that makes use of Perl 6's new features, which sets the stage for framework authors and application authors to start making use of Perl 6. At that point we'll need to see the Perl community produce something truly spectacular, much like Ruby on Rails was to the Ruby community several years back. Once that happens, we can consider the Perl community to have revitalized itself.

Although it's not a new concept by any means, the use of JavaScript for server-side development has gotten some attention recently. This is unfortunate, as JavaScript is not the sort of technology you want to use when developing the back-end for a Web site.

Even as a client-side language, JavaScript has proven to be of a very questionable quality. I think Jonathan Edwards was correct when he described JavaScript as being "quick" and "dirty", and only just "good enough" for most users. For years now, even the major JavaScript implementations have exhibited horrible performance. Only after 15 years of use are we seeing that start to change, with faster VMs from Adobe, Mozilla and Google. As a language, JavaScript itself does very little to encourage the development of secure, high-quality code.

Server-side development calls for a more rigorous programming language and environment than JavaScript can offer. Considering JavaScript's performance problems on the client-side, it's likely they'd be magnified when it comes to server-side development. And JavaScript really doesn't offer anything beyond more traditional server-side languages like Java, C#, PHP, Perl, Python and Ruby. As is mentioned in the Ajaxian article, there is a huge amount of basic Web back-end infrastructure that will need to be built to make server-side JavaScript even barely comparable to any of the aforementioned programming languages.

What's worse, however, is the general "attitude" associated with JavaScript. It has always been a language that appeals to those who want to get their scripts written quickly, even if the quality is terrible and there are major security flaws. It has often been used by Web designers to quickly add some interactivity to the pages they have designed, without needing an extensive programming background. Indeed, this amateurish attitude becomes very dangerous on the server.

PHP is a good example of such danger. For years many PHP developers generated SQL queries directly using data sent to their script by the user, with little to no filtering. This resulted in SQL injection attacks becoming very common for PHP-based software. We still even see this sort of development happening today, in some very sad cases. Meanwhile, more mature server-side languages and environments like Java and .NET have for years provided support for parameterized SQL queries. But even the most inexperienced of Java and .NET developers know better than to not appropriately filter user input.

While there will of course be some professionals using it, and using it well, I fear that the rise of server-side JavaScript would lead to nothing but problems. Just as the PHP community is starting to get its collective act together, we might see a new generation of inexperienced, amateurish developers start with server-side development using JavaScript.

Then again, maybe that's not a bad thing. It'll guarantee a steady stream of work for those of us who have made a career out of fixing such mistakes. But it is unfortunate that we couldn't stop such problems in the first place, but using better languages for server-side development. With some work, Haskell could become very useful for server-side development where quality is concerned, while Erlang plays its part in reliable and highly-scalable systems. Regardless, the main thing we need to keep in mind is that JavaScript should be avoided for server-side development.

I just finished reading this account of a developer who spent over an hour trying to track down an unexpected application crash that was eventually determined to be due to a tuple being passed to a function that was expecting a list. This incident happened while using Erlang. For certain types of systems, Erlang is a fantastic language and platform to use. In the coming years, it is likely that its model for handling concurrency will become familiar to most developers. Unfortunately, for all of its benefits, Erlang does have some drawbacks. One of the most significant is that it is a dynamically typed language.

The main problem with dynamic typing is that it allows incidents like the one described by Travis. A simple typo, in his case involving only two characters, can quickly become an hour or more of wasted time. Were static typing, such as that offered by OCaml or Haskell, being used, the problem would have been detected immediately.

Using OCaml (via its interpreter) as an example, we'll define a function that takes a list of strings, and prints each one to stdout:
    $ ocaml
        Objective Caml version 3.11.0

    # let f = List.iter (Printf.printf "%s\n");;
    val f : string list -> unit = <fun>


Next, let's see what happens when we accidentally call it with a tuple of strings:
    # f ("test", "test", "test");;
    Error: This expression has type string * string * string
        but is here used with type string list


It clearly has detected the mistake, and lets us know right away that there's a problem. Better yet, it tells us exactly what the problem is: it expects the argument to that function to be a list of strings, rather than the tuple of three strings that we passed it. Now that we have so quickly identified our mistake, we can go ahead and fix it. Passing in a list of strings has the expected result:
    # f ["test"; "test"; "test"];;
    test
    test
    test
    - : unit = ()
    #


We would have gotten similar results were we using the OCaml compiler rather than the interpreter.

Using a language that offers static typing is clearly the better route to take when we care about saving developer time, writing quality software, and detecting potential bugs and problems as soon as possible. It's unfortunate that Erlang is a dynamically typed language, as static typing would help increase the quality of code written using it even further. And it would've caught a minor mistake that has caught at least one developer over an hour of time.