Pinderkent

I read an article today about whether higher-level programming languages like Python, Perl and C# are really that much more productive than a comparatively lower-level language like C. This is not a new line of discussion, by any means. But we're getting to the point where we've been using such higher-level languages for over a decade, and thus have had more of an opportunity to observe and analyze how successfully (or not) they've been used.

In my view, that article comes to the general conclusion that many of the popular claims regarding the benefits of high-level languages versus low-level languages don't hold true. It's suggested that languages like C do indeed have aspects that hamper developer productivity, but high-level languages bring their own, albeit different, set of problems. While a C developer may run into problems with pointers, a Perl programmer might lose a similar amount of time optimizing regular expressions.

I think the conclusions of that article are correct to some extent, but I also think that the greater picture may have been missed. The real impact of languages like Python, Ruby, Perl, JavaScript and PHP isn't that they allowed expert programmers to be marginally more productive. Their greatest "benefit" (or arguably their greatest disadvantage) is that they have allowed average and even poor programmers to accomplish things they couldn't have reasonably done in C.

PHP and JavaScript are good examples of this. As anyone who has used them knows, they are very unremarkable languages. Conceptually and syntactically, they're much like C in many ways, but without some of the aspects of C that average developers often find bothersome. They aim to eliminate manual memory management, for instance. They offer slightly nicer string handling. Their execution environments aren't as tied to the native hardware. But otherwise, the core PHP and JavaScript languages generally offer the same basic functionality that C offers.

By eliminating some of the more difficult aspects of C, even if they're not as flexible or as powerful in many ways, they've made programming accessible to people who otherwise would not have been able to handle it. I've had the misfortune of working with people like this. They can understand the concepts of variables, constants, loops, conditionals, functions and even the basics of OO to some extent. But they're totally unable to understand some of the basic, yet essential, concepts of C. Pointers, for some reason, is a common one. But luckily for such average developers, languages like PHP and JavaScript make their lives easier by getting rid of such constructs and functionality.

So we soon enough see these average developers using languages like PHP and JavaScript to develop applications. In many cases, there's little to nothing preventing the same application from having been developed using C, aside from the inability of the average developer or developers to use C. Anyone who has worked in the industry knows why businesses opt to go with such solutions. Sometimes it is cheaper and easier to hire several PHP and JavaScript developers, instead of just one or two expert C developers. Other times it's because inexperienced or unknowledgeable managers just don't know any better, or have bought into hype and marketing. Regardless, the outcome is typically a system that just barely works, assuming it's not outright broken. Whatever costs might have been saved initially end up becoming far more costly in the long run.

Had those JavaScript and PHP developers been forced to use C, it's likely that we wouldn't have seen any sort of a software system be produced at all. They would've still been struggling with significant memory leaks, segfaults, and sometimes even just getting their code to compile. So we can see the actual main benefit of such higher-level languages; they've reduced the complexity of an otherwise difficult skill down to something that is more palatable to non-experts.

Now, we need to ensure that we don't lump high-level languages like Haskell, Erlang, and Common LISP in with other high-level languages like PHP and JavaScript. They are clearly very different. Haskell, Erlang and Common LISP, for instance, use abstraction to empower the developer. They offer advanced features and techniques that expert developers can build upon to great benefit. This is very different from languages like PHP and JavaScript, which clearly took the C model of computing, and stripped out the parts that make C more awkward for the less-skilled programmers.

Even thought they are significantly higher-level languages than C, languages like Erlang, Haskell and Common LISP haven't become as popular because they still require a high level of knowledge and expertise to use for even the most basic of tasks. So they highlight the important difference between a language being "high-level" and a language being "accessible". Functional languages increase expert programmer productivity with more powerful abstractions; PHP and JavaScript increase average programmer productivity via simplification.

The whole debate with respect to whether high-level languages are better than low-level languages will likely rage for many more years. There are some tasks that just can't be done in languages like JavaScript and PHP, so we will surely see C remain around for a long time. But we likely will see languages like PHP and JavaScript remain around for a similar reason. Unfortunately, that reason won't be about allowing good developers to develop more advanced software more quickly, but rather about letting poor developers continue to put out just barely suitable software systems.

An XML-like syntax is suitable for marking up other data. XHTML is a good example of an appropriate use of XML. There are numerous data transmission formats that make use of XML rather effectively. Even configuration files, to some extent, can be represented via XML. But one thing that should not be represented via an XML-like syntax is logic of the sort found in programming languages.

X# is a good example of why XML-based programming languages aren't a very good idea. One look at their X# sample programs should make it obvious why this is true. Any of those tasks could have been performed using a language like Perl or Python. But the Perl or Python equivalents would have been shorter, easier to read (and thus easier to maintain), and more portable. Many people often comment that Common LISP and Scheme have an awkward syntax to work with, but XML-based programming languages take that awkwardness to the next level, often without the benefits that LISP and Scheme do offer.

XML is best left as something a machine should process. It's not well-suited for humans to be reading and manipulating more than need be. It's especially not a good syntax to use for a programming language, especially one that offers non-declarative, imperative features. So I hope I never encounter XML-based programming languages like X# in practice.

It's not the end for Perl. It hasn't died. We won't see that happen for some time, if ever. But lately, it sure has lost its way. Andrew Binstock recently pointed this out in his article about Perl. Although it is very difficult to measure or quantify a programming language's popularity, we can get an overall sense of the "buzz" surrounding it.

Perl used to have that community "buzz". About 10 or 12 years ago, when somebody brought up Perl in a newsgroup or mailing list discussion, there was genuine excitement about what it had to offer. Mainly, that was rapid development, CPAN, powerful regular expressions, and the ability to write CGI scripts, all with little expense. We had needs, and Perl did a fantastic job of meeting them.

Times have changed. We still need to be able to develop software rapidly and with as little cost as possible. But we now have to make better use of our systems with multicore CPUs. Languages like Erlang often do this better than Perl does. Also, we need languages that allow for applications to be more easily maintained. For most sizeable, real-world development projects, Perl code can quickly become a maintenance headache.

Furthermore, the main benefits of Perl are now offered by most other mainstream languages. Python and Ruby offer very good regex support, a suitably large library, support for rapid development, a high degree of portability, and a low cost. Python, for instance, often encourages the development of more maintainable programs than Perl does.

There's very little that'll draw people specifically towards Perl. Recently, I've seen it used only because that's what the developers were most familiar with, or in numerous businesses, it's what they're stuck maintaining. Few people these days seem to choose Perl because it offers something they just can't get elsewhere.

Andrew makes some good observations in his post. One we really need to consider is how, for example, the Python community has gone out of its way to create several usable and actively-developed implementations of the language. We have the CPython implementation. There's IronPython for .NET. There's also Jython for Java. And we can't forget Stackless Python.

We just don't see that sort of activity in the Perl community. Sure, there is Rakudo. And there is also the now-stagnant Pugs implementation. But neither is really suitable for production use. Meanwhile, the different Python implementations are used successfully by many different people and companies. The Python implementations have momentum; Perl does not.

I read a number of blogs, mailing lists, newsgroups, and online forums dealing with Perl. Likewise, I talk often with people in industry who make use of Perl on a daily basis. And one thing I've noticed is that there's little excitement to be found. I think the extraordinarily long time it has taken to get anything useful in terms of Perl 6 has been extremely harmful to the Perl community. To really see a programming language prosper, it needs that raw community excitement. Python and Ruby currently have this excitement. Erlang and Haskell have it, too. Even the Common Lisp community seems more up-beat than the Perl community.

It's far too early to label Perl as "dead", or even "dying". Perl will be with us for ages. But we shouldn't deceive ourselves into thinking that it still has the same momentum that it used to, or has momentum comparable to what we see in the Python, Erlang, Ruby and Lisp communities. The action we see within the Perl community pales these days compared to what is happening in the other communities.

Although it hasn't (yet?) caught on much in industry, anyone who follows modern computing trends will no doubt have heard of D. Its Web site describes it quite nicely: D is a systems programming language. Its focus is on combining the power and high performance of C and C++ with the programmer productivity of modern languages like Ruby and Python. Special attention is given to the needs of quality assurance, documentation, management, portability and reliability. The D language is statically typed and compiles directly to machine code. It's multiparadigm, supporting many programming styles: imperative, object oriented, and metaprogramming. It's a member of the C syntax family, and its appearance is very similar to that of C++.

There exists a listing of the language features of D. One thing we should notice rather quickly is how it acts as a solid bridge between the worlds of procedural, object-oriented and functional programming.

Although it may not have the rigorous mathematical backing of a functional language like Haskell, it does bring over many of the most practical features. For instance, it offers garbage collection, closures, nested functions, type safety, lazy function argument evaluation, compile-time function evaluation, generics, implicit type inference, and a lot more.

But unlike Haskell, it also offers a C-like syntax that is no doubt familiar to a great number of C, C++, Java and C# developers. This is where a lot of D's power lies. It brings many of the benefits of functional programming, while also allowing developers to continue to use the syntax they're familiar with.

A number of people within the functional programming languages community say that syntax is irrelevant; the semantics are what truly matter. That's a shallow view to take. When it comes to developing large software systems in a rapid manner, reuse becomes a necessity. That includes not only code reuse, but also the reuse of previously-acquired skills, such as the quick comprehension of code with a C-like syntax.

It's becoming clear that we will need to start making use of some of the features traditionally offered by functional languages if we want to continue developing more complex software. D will surely allow this to happen, likely to the degree initially necessary to truly realize the benefits of such an approach. But we have to ask, will a transition to purely functional languages ever be necessary if languages like D offer the functional features in a manner that is sufficient for most developers?

When considering the software development landscape at this point, I'd have to say that many developers will be swayed towards D, rather than going all the way to Haskell, Standard ML, OCaml, Erlang, Common Lisp, or a host of other functional languages. D may very well offer the best trade off between code/skill reuse and productivity increases from functional language features.

If D keeps improving as it has been as of late, we may very well begin to see its adoption on a far greater scale. It took a number of years for languages like Ruby and Python to really start to take off. D may be just around the corner.

Earlier today I was reading an article about Parrot. Parrot is, as stated on the project's Web site, "a virtual machine designed to efficiently compile and execute bytecode for dynamic languages. Parrot currently hosts a variety of language implementations in various stages of completion, including Tcl, Javascript, Ruby, Lua, Scheme, PHP, Python, Perl 6, APL, and a .NET bytecode translator."

So Parrot does sound like an interesting piece of technology. It's understandable how a common runtime for scripting languages could prove beneficial. But will it ever be a platform suitable for serious, production usage? I have my doubts.

Parrot has been under active development for quite some time now. The initial 0.0.1 release was made on September 10, 2001. During 2007, we've seen a release every month or so. So a lot of effort has been put into Parrot over the past six years. It has surpassed one of the major stumbling blocks with many Open Source projects, in that it has managed to build at least some development momentum. Unfortunately for its supporters, Parrot has never really seemed to catch on. I think there are a number of reasons for this.

Stability is probably the first problem. I don't mean "stability" in terms of the runtime crashing, or anything of that sort. I'm talking about "concept" stability. There has always seemed to be a relatively large amount of change between releases. While this is good, in that there are improvements being made and new ideas being implemented, this causes problems for users who want to build reliably upon Parrot. Individuals and businesses often do not, or cannot, invest the time and effort to track a continually-moving target like Parrot.

The language implementations for Parrot, while many in number, have been of limited use. Looking at the status messages of some of the most promising and practical language implementations shows why this might be the case. Such messages include:

• "Incomplete - but all examples and test cases are working." (Amber for Parrot)
• "Most of the samples work." (BASIC/compiler)
• "Has been broken for a long time." (BASIC/interpreter)
• "Parser is pretty complete. Generates PIR for basic Ruby programs" (Cardinal, Ruby CVS Head 1.9 implementation)
• "Functioning, all samples working, lacks IO routines" (Cola)
• "Working for some simple forms. Due to some broken features, most of the bootstrapping code has been commented out." (Common Lisp)
• "Functioning for handcrafted test cases. Loading frozen state is currently broken. Far from complete." (Parrot m4)
• "This project has been abandoned. Any takers?" (Pint, an experimental PHP implementation)
• "Passes nearly 25% of tcl's (lightly converted) test suite, using a Test::More like harness." (Tcl)

So while there are many interesting language implementation projects for smaller or more obscure languages that have reached further stages of completion, the ones that were most likely to be of practical use seem to be lacking. Now, this is understandable. Maintaining a suitably complete Ruby, Python, Perl or Tcl implementation is not an easy task at all! And even if it were possible to come up to speed with the existing implementations, just staying current would be a major undertaking.

The third issue is that Parrot never really seemed to get a powerful backer. Java has Sun. .NET has Microsoft. Mono has Novell. Languages like Python, Perl and Ruby have large user communities, plus numerous business and academic supporters. While Parrot does have a unique community, as a whole I don't know if it has the resources necessary to put together and maintain an effective, usable, multi-language runtime environment.

Having followed the project for some time, I have come to realize that it may never really amount to anything of practical use. Now, that's not to say it's useless. Putting it together has no doubt been an enjoyable task for many of its contributors. I imagine a small number of people have found a use for it. While it would be nice if it were to grow beyond that, it's not something I expect to happen. It just doesn't offer what most serious users need.