ned Productions Consulting


Technology musings by Niall Douglas
ned Productions Consulting
(an expert advice and services company based in Ireland)


Friday 17th February 2017 3.44pm

Link shared: https://pypi.python.org/pypi/pcpp

I've been a little quiet on here of late since I came out of contract, and it's amazing how five weeks of unemployment have passed just like that. I had to relinquish my rented office inside town to conserve cash a little over two weeks ago, and since I've returned to being at home all day I've essentially become mostly a daycare worker. It's no bad thing, Clara gets taken out each day to do something with her Daddy, and she's very, very pleased about that.

Since I've returned home, I've also changed things up on my open source work partially because I was getting sick of looking at proposed Boost.Outcome and that never ending tail of small boring things which must be done before submission to Boost peer review. So for the past two weeks or so I've been immersed in a brand new side project written in Python which has proven very useful in refreshing my Python skillset, which had languished a little in recent years. Thus marvel and be amazed at my newest creation, pcpp (https://pypi.python.org/pypi/pcpp) which is a C99 preprocessor written in pure Python, written during Clara's daily naps and when I put her to bed.

It's not fully debugged yet but nearly there. And it solves a very long standing problem I've had for many years now, namely, how do I make "drop in and go" editions of my C++ libraries? The thing which had put me off writing a C preprocessor was mainly the hassle of getting function-like macros working properly, after all Microsoft's compiler famously has a very broken implementation of those in their preprocessor which has caused never ending woe for anyone on that platform for two decades at least. So getting them right isn't trivial, and indeed as expected my Python implementation saw rapid and highly standards conforming development right up until I began the function-like macros which has demonstrated to me how ambiguously the C and C++ standards are written. Still, I'll get there eventually.

And it's really nice to be back programming in Python! It's just so amazingly good at all the stuff C++ isn't, whilst also being strong in all the areas C++ is also strong in like handing absolute untrammelled power to the programmer and strong type and class language support. And, yet again, and to the continuing surprise of many in the C++ community, pcpp demonstrates that if you know what you're doing, Python programs are not slow. As with writing high performance C++, you need to understand very well how the bare metal works, and once you do, you can write surprisingly high performance code in Python. As with C++, a lot of the time you need to ignore 101 compsci algorithmic theory because the Python interpreter muddies that up enormously. So pcpp for example does lots of stuff like repeated totally unnecessary string searching which would be performance suicide in C++, but is far faster and scales far better than more compsci pure approaches which would cause Python to instance non-trivial objects, something you want to avoid in fast executing Python.

Of course, modern CPUs are increasingly little different in deviating substantially from compsci algorithmic purity, it's just a difference of degree. Knowing when and how to write theoretically terrible code is the mark of the experienced expert. I'm no guru level Python programmer, but then I'd say I'm no guru level C++ programmer either. I'm just keen on sufficient performance and good scalability, whatever the programming language I use.