Software Quality and Testing

• Software Engineering: Dead? – “And yet, it’s also a release. It’s as if a crushing weight has been lifted from my chest. I can publicly acknowledge what I’ve slowly, gradually realized over the last 5 to 10 years of my career as a software developer: what we do is craftsmanship, not engineering. And I can say this proudly, unashamedly, with nary a shred of self-doubt.”
• The four essential laws of software updates – “Automatic updates are becoming more and more common, which is a good thing. They save us time and hassle: instead of finagling with installers, we can just sit back, relax, and let all our programs do the work for us. Oh, if only it were that easy.”
• Good Lazy and Bad Lazy – “Something which I’ve come to realise more recently is that it’s not necessarily true that being lazy as a developer is always a good thing – it depends in what way you are being lazy because there are certainly good and bad ways in which you can express your laziness!”
• Book Review: The Art of Unit Testing – “Given my personal commitment to software quality through testing, I was excited to get my hands on a copy of Roy Osherove’s The Art of Unit Testing from Manning. A few weeks ago our copy was delivered and I have been excited for an upcoming vacation which would give me the time to sit down and churn through the book.”

Performance and Scalability

• My Thoughts on NoSQL – “In the late nineties and early thousands, websites were mostly read-only–a publisher would create some content and users would consume that content. [..] And with that fundamental shift away from read-heavy architectures to read/write and write-heavy architectures, a lot of the way that we think about storing and retrieving data needed to change.”
• Latency is Everywhere and it Costs You Sales – How to Crush it – “Latency matters. Amazon found every 100ms of latency cost them 1% in sales. Google found an extra .5 seconds in search page generation time dropped traffic by 20%. A broker could lose $4 million in revenues per millisecond if their electronic trading platform is 5 milliseconds behind the competition. “
• Scalability issues for dummies – “Every once in a while I get people asking me what’s taking me so long to open my startup Inkzee to the public. They also ask me what exactly have I been doing as the web seems exactly the same. I normally answer that things aren’t easy, that it takes time, specially if you are alone, like I am. After a while I end up explaining my problems with scalability and that’s the point where people just can’t follow you. I’m going to explain here what are scalability problems and how deep the repercussions are for a small company.”

Process and Methodology

• Considering Agile, deciding against it – “Some time ago, while managing a distributed software project, I spoke to a developer who was thinking about joining our team. This individual was a big proponent of Agile development. When he found out that in our particular project we were not using an Agile development model, he began to lose interest. He compared my position there to the position of the Tom Smykowski character from the movie Office Space.”
• Planning for Feature-Complete Deadlines – “Planning to catch a plane is a perfect example of risk management for contained change: problems are expected, but they usually fall within a range you can still make an estimate of. (I wrote about this in my post The Three Types of Change.) Using experience I can make a probabilistic forecast of the time required to catch the plane. It will take two hours at the least to be there on time. But when Murphy’s Law hits me (and it usually does in some way) it could be anywhere up to four hours.”

Usability and Accessibility

• Mobile Usability – “When designing for mobile, there’s a tension between (a) making content and navigation salient so that people do not work too hard to get there, and (b) designing for a small screen and for slow downloading speeds. That’s why almost every design decision must be made in the context of the site being designed, and what works for a site may not work for another. “
• The Inclusion Principle – “Affordance allows us to look at something and intuitively understand how to interact with it. For example, when we see a small button next to a door, we know we should push it with a finger. Convention tells us it will make a sound, notifying the homeowner that someone is at the door. This concept transfers to the virtual environment: when we see a 3D-shaped button on a web page, we understand that we are supposed to “push” it with a mouse-click.”
• Reinventing the desktop (for real this time) – Part 1 – “Since the 1980’s, this functionality has been presented to users on most systems with only minor variations upon the standard WIMP (Window, Icons, Menu, Pointer) model handed down from Xerox PARC and the first Mac, so, obviously, the modern desktop is not really broken: people have been getting by with essentially the same design for decades now. Still, there is a perennial longing for something better, so the question is what motivates this feeling?”

Security

• Hacking CSRF Tokens using CSS History Hack – “In this exploit, we discover the csrf token by brute forcing the various set of urls in browser history. We will try to embed different csrf token values as part of url and check if the user has visited that url. If yes, there is a good chance that the user is either using the same CSRF token in the current active session or might have used that token in a previous session. Once we have a list of all such tokens, we can just try our csrf attack on the server using that small list.”

Software Quality and Testing

• Forgotten Refactorings – “I don’t know how much more emphasized step 1 of refactoring could be: don’t touch anything that doesn’t have coverage. Otherwise, you’re not refactoring; you’re just changing shit. And not a single IDE enforces or encourages you to have test coverage before mucking about with the refactorings.”
• Benefits of automated functional testing – “In the last few years of writing “enterprise” software in Java I’ve created and maintained thousands of unit tests, often working on projects that require 80% or better code coverage. Lots of little automated tests, each mocking out dependencies in order to isolate just one unit of code, ensuring minimal defects and allowing us to confidently refactor code without breaking everything. Ha! I’ve concluded that we’d be better off deleting 90% of these tests and saving a lot of time and money in the process.”
• Designers who wireframe: pros and cons – “There are plenty of conflicts of interest inherent in running a small creative shop where everyone has to wear lots of hats (or where one person wears all the hats)—like the account manager ideally should not be responsible for quality control because there’s too much conflict between the need to keep the job within budget and the need to take the time to do things right.”

Performance and Scalability

• SQL Databases Don’t Scale – “A question I’m often asked about Heroku is: “How do you scale the SQL database?” There’s a lot of things I can say about using caching, sharding, and other techniques to take load off the database. But the actual answer is: we don’t. SQL databases are fundamentally non-scalable, and there is no magical pixie dust that we, or anyone, can sprinkle on them to suddenly make them scale.”
• No to SQL? Anti-database movement gains steam – “One of the requirement for this new service was the need to build a graph that includes friends of friends, and products in catalogs. The process for building that graph with an In-Memory-Data-Grid took 2-3 msec vs. tens of seconds with a traditional approach (note that part of the complexity in this case is that the query itself is ad-hoc and can’t be easily partitioned). Building that graph on-the-fly at these speeds just couldn’t be done with traditional SQL database.”

Process and Methodology

• The problem with BDUF is the ‘B’ not the ‘UF’ – “I’m sure every developer with even the slightest amount of experience has run into a situation where a proposed design for a domain concept is initiated, and then someone (perhaps even the developer in question) has decided something along the lines of: “We need to generalize this so that it covers future requirements.””
• Agile is Dead – “The agile revolution has won. There is no doubt about it. No longer are its evangelists the pioneers who blaze new trails and chart the unknown, like they used to a couple of years ago. Agile principles, methodology, tool sets, even agile lingo, have become mainstream. Frankly, everybody and their dog is “doing agile” these days. Of course, there are pockets of resistance in the more backward (or “conservative” to be more polite) companies, but they are destined to eventually see the error of their ways and switch over.”

Usability and Accessibility

• Building Respect for Usability Expertise – “Respect comes only from proven performance. Once content owners see how much better customers react to websites that are written and designed according to established usability guidelines, they’ll start respecting you more. Sadly, this is a chicken-and-egg situation: you get to demonstrate the value of your advice only if it’s being implemented. “
• My first experience using an accessible touch screen device – “And I must say this was an amazing experience! My fingers definitely need to get used to gestures such as flicking or tapping, or using a rotor. But having an iPod Nano 4th generation helped with that, since moving the finger over the screen like on a dialer felt most like tracking around the iPod’s click wheel. Even the sound the rotor makes is the same. ”

Technical Tips

• Low Level Bit Hacks You Absolutely Must Know – “Bit hacks are ingenious little programming tricks that manipulate integers in a smart and efficient manner. Instead of performing some operation (such as counting the 1 bits in an integer) by looping over individual bits, these programming nuggets do the same with one or two carefully chosen bitwise operations.”

Misc

• Subtractive Design – “Subtractive design is the process of removing imperfections and extraneous parts in order to strengthen the core elements. You can think of a design as something you build up, construct and let grow, but it’s pruning away the excess that gives a design a sense of simplicity, elegance, and power.”
• Getting Pretty Lonely – “WordPress is licensed under the terms of the Gnu Public License (GPL) which, in a nutshell, stipulates that you are free to use the software however you like, but if you make changes and distribute those changes, then you must share those changes under the same terms. This simple, radical restriction means that you are prohibited from taking a GPL project and incorporating it with a closed-source project.”
• Making Games – “I hear from programmers every now and then who are considering trying to get into the games industry. Our projects are strange beasts that are similar to other software projects in some ways, yet vastly different in other ways. Still, if you look at it from outside, most of these differences will be hard to spot.” and a follow-up.

Related posts:

1. Software Quality Digest – 2009-07-27

Software Quality and Testing

• The Case for Multiple DBs in Multi Tenancy situations – “There are two major, reasonable approaches that I know of: First, to have a database per tenant. (This is the approach discussed by Ayende in his multi tenancy series) A second common alternative is to have a tenant id column in all tables that require differentiation by client. My team was in a situation requiring multitenancy, and needed to select an approach. Wary of the idea of multiple databases, the team immediately decided to go with the tenant column approach.”
• API Design Matters – “After more than 25 years as a software engineer, I still find myself underestimating the time it takes to complete a particular programming task. Sometimes, the resulting schedule slip is caused by my own shortcomings: as I dig into a problem, I simply discover it is a lot more difficult than I initially thought, so the problem takes longer to solve—such is life as a programmer.”
• Is the Supremacy of Object-Oriented Programming Over? – “I never expected to see this. When I started my career, Object-Oriented Programming (OOP) was going mainstream. For many problems, it was and still is a natural way to modularize an application. It grew to (mostly) rule the world. Now it seems that the supremacy of objects may be coming to an end, of sorts.”
• Java vs .NET Developers – “Much of what we discuss in .NET is far more common in Java as Java has a far healthier community than Microsoft. You will find terrible developers on both side but the terrible developers on the Java side have more mainstream exposure to good ideas than on the Microsoft side.”
• The difference between Infrastructure & Application – “Recently I am finding myself writing more and more infrastructure level code. Now, there are several reasons for that, mostly because the architectural approaches that I advocate don’t have a good enough infrastructure in the environment that I usually work with. Writing infrastructure is both fun & annoying.”
• A Crash Course In Failure – “When is the last time you intentionally unplugged a live, production server? Better still, when is the last time you intentionally unplugged a rack of live, production servers? I can think of a couple of reasons why the answer might be ‘never.’”
• To Spec or Not to Spec – “There’s been much debate of late regarding speculative or ’spec’ work and the associated problems (and benefits) of it. [..] But what exactly is spec work? Some people don’t seem to know. Why all the hoo-ha and what, if anything, should we do about it?”

Performance and Scalability

• Needle in a haystack: efficient storage of billions of photos – “The Photos application is one of Facebook’s most popular features. Up to date, users have uploaded over 15 billion photos which makes Facebook the biggest photo sharing website. For each uploaded photo, Facebook generates and stores four images of different sizes, which translates to a total of 60 billion images and 1.5PB of storage. The current growth rate is 220 million new photos per week, which translates to 25TB of additional storage consumed weekly. At the peak there are 550,000 images served per second. These numbers pose a significant challenge for the Facebook photo storage infrastructure.”
• GZIP encoding = happier users? – “I guess there are very few cases where you shouldn’t use gzip your content. If your typical page is less than 100 bytes then gzipping it could hurt the client’s and the server’s performance. But no website —except maybe a few web-services— serves pages with a typical size of 100 bytes or less. So there’s no excuse for serving uncompressed HTML.”
• Use Thread Pools Correctly: Keep Tasks Short and Nonblocking – “Thread pool tasks should be as small as possible, but no smaller. One reason to prefer making tasks short is because short tasks can spread more evenly and thus use hardware resources well. [...] The larger the tasks, the more unwieldy the pool’s workload is, and the harder it will be to spread the work evenly across the machine all the time. On the other hand, tasks shouldn’t be too short because there is a real cost to executing work as a thread pool task.”

Usability and Accessibility

• In Defense of Eye Candy – “We’ve all seen arguments in the design community that dismiss the role of beauty in visual interfaces, insisting that good designers base their choices strictly on matters of branding or basic design principles. Lost in these discussions is an understanding of the powerful role aesthetics play in shaping how we come to know, feel, and respond.”

Technical Tips

• The Stack Is An Implementation Detail – “Almost every article I see that describes the difference between value types and reference types explains in (frequently incorrect) detail about what “the stack” is and how the major difference between value types and reference types is that value types go on the stack. I’m sure you can find dozens of examples by searching the web.” and part 2
• Validation – Part 1 – Getting Started – “There are a lot of good and free frameworks to help you deal with validating user-submitted data. [..] Despite these frameworks, there may be situations where you need to roll your own, which is what we’re going to look at in this series. We won’t be doing anything fancy, but we will look at an end-to-end custom solution.” part 2 and part 3
• The Mystery Of CSS Sprites: Techniques, Tools And Tutorials – “CSS Sprites are not new. In fact, they are a rather well-established technique and have managed to become common practice in Web development. Of course, CSS sprites are not always necessary, but in some situation they can bring significant advantages and improvements – particularly if you want to reduce your server load.”
• Is ASP.NET MVC a half-baked solution? – “Let’s be clear, ASP.NET MVC is an improvement over WebForms – and those developers refusing to leverage ASP.NET MVC (or something else) for new projects are simply lazy. WebForms is full of leaky abstraction, really tight coupling, and ridiculous concepts – such as viewstates and postbacks.” and a follow-up
• I Spose I’ll Just Say It: You Should Learn MVC – “WebForms is a lie. It’s abstraction wrapped in deception covered in lie sauce presented on a plate full of diversion and sleight of hand. Nothing you do with Webforms has anything to do with the web – you let it do the work for you. This, friends, is a big deal (at least to me): You’re working in a lie.”
• Two techniques for faster JavaScript – “I like things that go fast, and that includes code that runs fast. With JavaScript (and Java, too), that can be a challenge. So much the better, though. I like challenges too.”

Misc

• Has The Virtualization Future Arrived? – “The attraction of virtualizing older operating systems is that it throws off the eternal yoke of backwards compatibility. Instead of bending over backwards to make sure you never break any old APIs, you can build new systems free of the contortions and compromises inherent in guaranteeing that new versions of the operating system never break old applications. “
• Time and Space Tradeoffs in Version Control Storage – “Storage is one of the most difficult challenges for a version control system. For every file, we must store every version that has ever existed. The logical size of a version control repository never shrinks. It just keeps growing and growing, and every old version needs to remain available. So, what is the best way to store every version of everything?”

Related posts:

1. Software Quality Digest – 2009-07-27

Software Quality and Testing

• Turn off your step-thru debugger – “My mentor suggested a couple more approaches to try (and when I tried them, they worked, of course). Then he made a remark that has stayed with me ever since. ‘I try to stay away from debuggers,’ he said. ‘A debugger is a crutch. You’re better off without it.’”
• Exception-Driven Development – “When a user informs me about a bona fide error they’ve experienced with my software, I am deeply embarrassed. And more than a little ashamed. I have failed to see and address the issue before they got around to telling me. I have neglected to crash responsibly.”
• “I don’t have time to test!” – “Now, I think that abandoning testing due to time constraints is reckless and irresponsible. I sometimes catch myself thinking that I don’t have time for test writing, but I soon find myself proving this argument wrong. Often people who don’t have time to test will waste hours debugging code that they are desperately trying to force into production. This code ends up being brittle and costly to produce anyway.”
• Coupling and Cohesion – “One of the things about design that makes it such a joy is that it requires balance. If elements are too large, each change will be more expensive than it needs to be. If elements are too small, changes will ripple across elements. And optimizing the design takes place against the backdrop of an unpredictable stream of changes.”
• It’s Not TDD, It’s Design By Example – “The word “test” in software is a very loaded term. The first time I came across a tester (as in, a person performing the quality assurance role) was in my first professional job. We were never taught about testing at college, as we were all there to learn to be developers.”

Performance and Scalability

• Performance optimization: The first thing to do – “So, here is my point: The most important thing about optimization is analysis. You can’t fix a problem by simply trying different solutions to see if they work. In order to fix a problem, you have to understand the problem first.”
• Learned lessons from the largest player (Flickr, YouTube, Google, etc) – “If you working in new website; don’t buy too much equipment just because you want your website to be faster. Start slow, and small, and using the right statistics you will know the right direction. Measuring the right thing is fundamental and important. Selecting a benchmark and comparing the results seems, initially at least, to be a simple problem, but a host of mistakes are made during this process.”
• Experiences deploying a large-scale infrastructure in Amazon EC2 – ”
  You’ll say that you’ve done lots of load testing before. This is very good….but it still will not prepare you for the sheer amount of traffic that the internets will throw at your application. That’s when all the things I mentioned before — automated deployment of new instances, charting of the important variables that you want to keep track of, quick identification of bottlenecks — become very useful.”

Process and Methodology

• Why do our time estimates suck? – “Everyone dealt with it in some form of another. Either you have given your estimations and you missed it by a long shot or someone handed you some estimations and they grossly underestimated the amount of work. And when that mark is missed, its the wag of the finger and the rain cloud of shame upon you. Management has deemed you a failure. Way to go.”
• The Pond – “I cringe. It’s Ian and Ian is a senior engineer. He’s a rock. He gets it done. I never have to ask him twice and, after six years, Ian has every right to ask to work remote. But I’m still freaked because my first thought when anyone asks to work remote is, “This fine person is a year away from either quitting or being fired.” Why? Because they’re asking to leave the Pond.”
• Pair Programming: The Code Fairy – “One of the hardest situations that comes up when pair programming is when you want to solve a problem in a certain way but you can’t persuade your pair that it’s the approach you should take. The temptation in these situations is to wait until your pair isn’t around, maybe by staying late at the end of the day or coming in early the next day and then making the changes to the code that you wanted to make but didn’t when you were pairing with them.”

Technical Tips

• Constraint Yourself! – “Part of this epiphany is realizing that DDL (Data Declaration Language), DML (Data Manipulation Language) and DCL (Data Control Language) are all an intrinsic part of SQL and not disjoint languages that stand apart. This article deals with the DDL, but better DDL makes for better DML and DCL. When a business rule is in the DDL, it is done one way, one place, one time. You do not have to hope that every application and every DML statement gets all the rules right. You do not have to hope that a change in the rules will be corrected in hundred or even thousands of places in the system. “
• The Dangers of the Large Object Heap – “Usually, .NET developers don’t need to think too much about how their objects are being laid out in physical memory: after all, .NET has a garbage collector and so is capable of automatically removing ‘dead’ objects and rearranging memory so that the survivors are packed in the most efficient manner possible. The garbage collector has limits to what it can do, however; and when these limits are reached, then the runtime can exhaust memory in a way that is surprising and confusing to any developer who is not aware of how .NET chooses to lay out objects in memory.”
• 15 Essential Checks Before Launching Your Website – “Your website is designed, the CMS works, content has been added and the client is happy. It’s time to take the website live. Or is it? When launching a website, you can often forget a number of things in your eagerness to make it live, so it’s useful to have a checklist to look through as you make your final touches and before you announce your website to the world.”

Misc

• We Slice Software Vertically Just Like Bread – “Some tasks look like they’ll take weeks to complete. That my friend is not a task. We need to split that. There is probably some aspect of that which could be completed in a day.”

Related posts:

1. Software Quality Digest – 2009-07-27

We aren’t experts in all the different frameworks and platforms we evaluated and there are probably some details we got wrong. Feel free to correct us by leaving a comment. So without further ado, here are the pros and cons of the different development stacks in regards to installable web applications:

• Java (including Grails and Spring)

  Java and the various frameworks such as Spring and the dynamic counterpart Groovy combined with the Grails framework qualify in many ways to be the perfect web development stacks for installable applications. The Java platform is very mature and has very good support for cross-platform and cross-database development. If you use frameworks such as Spring, you also benefit from the MVC pattern and from enterprise features and libraries. Java also isn’t going anywhere and it’s safe to say that Java will be available for the next 10+ years and probably a lot longer. Unicode support is also excellent, which is rare for cross-platform development stacks.

  However, one of the biggest drawbacks of this platform also made us decide against using Java: deploying Java web applications is complicated and scares the hell out of us. We wouldn’t feel comfortable having to support deployment scenarios with all the different available Java application servers out there, which you need to run a Java web application. The need for an application server such as Tomcat or JBoss is a serious disadvantage, especially if your customers aren’t experienced with installing and configuring such a solution.

• Ruby on Rails

  Ruby on Rails is a modern and productive web development stack and framework. Using Ruby as the programming language of choice, it’s best known for the MVC architecture and the well-integrated ActiveRecord object-relational mapping system for database access. Ruby on Rails also emphasizes the convention over configuration concept, making it very easy to get started developing web applications. Rails also comes with reasonable support for multiple database servers, but is usually used with MySQL and PostgreSQL.

  Recent versions of Rails also improved the support for Unicode and UTF-8 encoding, making it more suitable for applications that are used by users from non-western countries. Rails is usually deployed by configuring a Mongrel cluster (which is easier than it sounds) and using a front-end web server such as Apache, nginx or lighthttp for static content serving and load balancing the different Mongrel processes. Although setting up a run-time environment for Rails applications is still a lot easier than for Java web applications, it can still be problematic for customers who have no experience with Rails and/or Unix environments.

  This leads us to the big problem with Rails: the support for Windows servers is miserable. Because most Rails applications are developed, tested and deployed in Unix environments, either for hosted web applications or in-house intranet applications, supporting Windows servers isn’t an important issue for the Rails team or developer community. However, if you are trying to sell an installable web application to typical companies, Windows servers are still extremely popular for internal application and workgroup servers. And if your potential customers are used to running Windows servers, you better make sure that your solution works flawlessly in this environment.

• ASP.NET

  ASP.NET is a good choice for many installable web applications. It’s reasonable easy to deploy an ASP.NET web application on a Windows Server with IIS and most companies (and potential customers) already have such a server running as an internal application server. The Unicode and localization support is also top-notch and if this is one of your most important requirements, you should definitely consider going with this development stack.

  Another big plus for ASP.NET is the strong developer community with countless of blogs, books, websites and forums. The rich offering of user interface controls, libraries and development tools for the .NET platform in general and ASP.NET in particular by third-party vendors is also unmatched. However, ASP.NET also has its problems. ASP.NET Web Forms really encourages mixing application logic with the user interface code, which can result in badly designed applications and hard-to-maintain code. The abstraction in ASP.NET Web Forms that’s supposed to make web development easier for Windows desktop developers makes the development stack hard to debug and overcomplicates things.

  Experienced ASP.NET developers try to workaround these limitations by using patterns such as MVP or MVC. Especially Microsoft’s new ASP.NET MVC framework is an interesting alternative to Web Forms, as it encourages the separation of business logic, data access/validation and user interface code and makes it easier to build a maintainable application. This leaves us with the biggest drawback of ASP.NET: the lack of cross-platform support. Sure, there’s Mono, an open source .NET implementation that also supports ASP.NET. But while it’s definitely a nice option to have, it’s not widely used and I personally would not consider it to develop cross-platform web applications at this point.

• PHP

  PHP is arguably the most used web development programming language, despite it’s many technical limitations and problems. The main reason PHP has become so popular is that it’s extremely easy to use and that it’s supported by virtually all web hosting companies. Developing an PHP application without using a decent application framework will most likely lead to an unmaintainable code base, however. The language itself, although explicitly designed for web development, doesn’t come with a (MVC) framework and doesn’t help with separation of concern in any way.

  One of the technical limitations is the lack of Unicode support. While most development stacks have quite good Unicode support by now, PHP’s support for non-western characters is still very limited and if you plan to support most server configurations and multiple databases, it’s basically non-existent.

  Luckily, there are quite a few available PHP frameworks to remove some of the mentioned limitations such as the Zend framework, CakePHP or CodeIgniter. Using one of these frameworks makes it a lot easier to write maintainable applications and also provides you with often needed functionality not found in PHP itself, such as a database abstraction layer.

  Of course, PHP also has its good sides. Applications written in PHP are very easy to deploy and work on most operating systems and server configurations. Even Microsoft is working hard to improve the PHP deployment experience on Windows and IIS, as many popular web applications such as WordPress are written in PHP. The great cross-platform, cross-database support combined with the ease of deployment and available frameworks make PHP a good web development stack, even though it’s not the best technical choice.

Having evaluated the above mentioned frameworks and having looked at the pros and cons of each development stack, we have come to the conclusion that PHP together with a lightweight framework such as CodeIgniter is the best choice for an installable web development stack, at least today. Yes, from a technical standpoint, PHP is less than great. The ability to easily deploy and run a PHP application on most server configurations, however, easily outweighs its limitations. One drawback that really made us think twice before deciding to go with PHP was the lack of proper Unicode support. Unicode support was originally a really important requirement to us, but we decided that we can do without Unicode support for a few more years until this problem will hopefully solve itself (with better Unicode support in PHP).

You might also be interested in our reasons to choose CodeIginiter as the PHP framework for our application. The great thing about CodeIgniter is that it’s a really simple framework. Although CodeIgniter only offers the most basic features and you have to implement advanced capabilities such as locale or authentication support yourself, this simplicity makes it really easy to understand the entire code base in a very short amount of time. So even in the unlikely case that we needed to fork this framework because the authors stopped supporting it, we could easily improve and add features to the existing code ourselves, something that is not really feasible with most other frameworks and stacks out there as they are often too complex.

Some months into development we are still happy with our choice. The application is coming along nicely and PHP and CodeIngiter have proven to be a solid development stack for an installable web application so far (of course, the real challenges come when customers deploy our application in many different hosting environments). Our choice of a development stack would most likely be different if we were building a hosted solution where we could control every little aspect of the server environment. But because our customers will have to deploy and host our application themselves, PHP and CodeIgniter are a great choice for us.

Now before you start your next big thing, make sure to consider all your options, as you cannot easily change platforms once you’ve invested a lot of time and resources.

Software Quality and Testing

• Intentional API Design – “Don’t underestimate the importance of having good APIs. And don’t assume that if the product design is good, APIs will take care of themselves. That’s like saying if an airplane’s design is good enough, it will fly itself. That’s almost true. The problem is the word ‘almost.’”
• Coding: It’s all about the context – “I’m not sure who said that ‘if you rewrite a system that you didn’t write the first time you’ll probably make the same mistakes as the original team did’ but it’s certainly becoming more clear to me how this would be the case.”
• Kill Your Darlings – “I’m not sure why so many developers become so attached to code they’ve written. Perhaps developers have the same attachment to code that artists have to their paintings or music. Or, perhaps they view their legacy code as job security. Whatever the reason, developers seem to have some illogical desire to extend the life of code they’ve produced.”

Performance and Scalability

• Performance Anti-Patterns – “Since software patterns are considered abstractions of positive experience, we can talk about the various approaches that led to these performance problems as anti-patterns—something to be avoided rather than emulated.”
• Art of scalability (series) – “At the simplest level, Scalability is about doing more of something. Scaling a web application is about allowing more people to use your application is same time. Scaling it’s the ability to handle larger number of concurrent users.”
• A rant on automatic parallelism – “I don’t believe the lion’s share of developers want to rearchitect and rewrite their code with parallelism at the forefront of their development process. They don’t want to think about shipping memory over to the GPU and launching a highly-specialized data parallel kernel of computation, nor do they want to have to add locks and transactions everywhere to make things safe. But I do, however, believe the lion’s share of developers wouldn’t mind if their code ran faster as hardware got faster (via more cores).”

Process and Methodology

• Pair Programming: Slowly but surely – “Software development for me is a lot more about thinking through our options than coding the first thing that comes to mind and pair programming helps drive this approach to problem solving.”
• Pair Programming: From a Lean angle – “I’ve worked on teams which pair programmed the whole time and teams where pair programming was less prevalent and the difference in how well the knowledge of the code base was spread throughout the team is massively different.”
• Game Development going Agile – “Game development differs from business application development in many ways. Performance truly is king and a game dev may write code considered bad form to a business dev. Games have a fixed life cycle; once released there is no new development. This is the opposite of the business apps that are [...]. Differences aside however, it is comforting to see that the same Agile methodology, culture, and process can work for both. “
• Continuous deployment in 5 easy steps – “It’s a process whereby all code that is written for an application is immediately deployed into production. The result is a dramatic lowering of cycle time and freeing up of individual initiative. It has enabled companies I’ve worked with to deploy new code to production as often as fifty times every day.”
• Stumbling Through Mediocrity – “Worse, with so many organizations interested in “being Agile” rather than “being great,” a whole industry is springing up around providing watered-down, non-threatening, non-boat-rocking (and non-functioning) Agile. If the point is to Be Agile, there’s no need for Agile to actually work. Sell a certificate and walk away. Everyone’s happy, right? Right?”
• Are Some Software Development Challenges Too Complex for Agility? – “When the concepts of agile software development are introduced to an organization, a common response is, “Sure — that will work with a simple application, but ours is a very complex ‘Enterprise’ system”. Could it be that core agile software development principles are too simplistic to tackle the really tough situations? I recently found myself in a position in which it appeared that this might be so.”

Security

• Anatomy of a Cross-site Request Forgery Attack – “A Cross-site request forgery attack, also known as CSRF or XSRF (pronounced sea-surf) is the less well known, but equally dangerous, cousin of the Cross Site Scripting (XSS) attack. Yeah, they come from a rough family.”

Usability and Acessibility

• Designing Drop-Down Menus: Examples and Best Practices – “As a general rule, most Web developers, especially usability enthusiasts, say it is bad practice to use drop-down menus because they are confusing, annoying and oftentimes dysfunctional. From a design standpoint, however, drop-down menus are an excellent feature because they help clean up a busy layout. If structured correctly, drop-down menus can be a great navigation tool, while still being a usable and attractive design feature.”
• 10 Creative & Rich UI & How to Create Them – “Sometimes it’s just amazing to see, which level of usability, legibility and visual appeal can be achieved using some basic design techniques. In fact, some talented web-developers manage to deliver powerful, functional and gorgeous web-design in “look-and-feel”-style, which is easy to use and nice to see.”
• 8 Simple Ways to Improve Typography In Your Designs – “Many people, designers included, think that typography consists of only selecting a typeface, choosing a font size and whether it should be regular or bold. For most people it ends there. But there is much more to achieving good typography and it’s in the details that designers often neglect.”

Misc

• On the Speed of Light, Innovation, and the Future of Parsing – “These are the very early days of computing. We have just started exploring the shore of a new continent, and it’s shocking, because many are already proclaiming that it’s over. But the good news is that there is plenty of unconquered territory, most of the settlers have their eyes covered, and they’re not even attempting to grab any of it.”
• Should Competent Programmers be “Mathematically Inclined”? – “On the other hand, I have not found in practice that programmers need to be mathematically inclined to become great software developers. Quite the opposite, in fact. This does depend heavily on what kind of code you’re writing, but the vast bulk of code that I’ve seen consists mostly of the “balancing your checkbook” sort of math, nothing remotely like what you’d find in the average college calculus textbook, even. “

Related posts:

1. Software Quality Digest – 2009-07-27

Software Quality and Testing

• Five Dollar Programming Words – “Why are these words worth five dollars? They’re uncommon words that have a unique and specialized meaning in software development. They are a bit off the beaten path. Words you don’t hear often, but also words that provide the thrill of discovery, that ‘aha’ moment as you realize a certain programming concept you knew only through experimentation and intuition has a name.”
• Making the debugger redundant – “It got me thinking about how the way that we can reduce the need to use the Debugger when writing code in imperative languages. Debugging is so boring and takes so long that spending large amounts of doing it both crushes the spirits and slows you down considerably.”
• Reassessing what the debugger is for – “Admittedly the name ‘debugger’ doesn’t really help us here as the name describes a tool that “helps in locating and correcting programming errors” which is all well and good but I think it should be one of the last tools that we turn to rather than one of the first.”
• JavaScript Testing Does Not Scale – “It’s become increasingly obvious to me that cross-browser JavaScript development and testing, as we know it, does not scale.”
• TDD: The Art of Fearless Programming – “Test-driven development is a discipline of design and programming where every line of new code is written in response to a test the programmer writes just before coding.”

Performance and Scalability

• How to write fast code – “Key takeaway: Don’t think in terms of making a slow piece of code run faster. Instead, think in terms of making it do less.”
• Twice-as-fast isn’t good enough – “In very few cases will your life be changed by a two-fold speedup of anything computer-related. If your business plan (for a new technology) is predicated on speed, and the promised performance boost is less than a factor of ten (a full order of magnitude), rethink what you’re doing. ‘Twice as fast’ is not a competitive advantage. Not by a long shot.”
• Google searches for holy grail of Python performance – “Google’s Python engineers have launched a new project called Unladen Swallow that seeks to improve the performance of the Python programming language. One of the project’s goals is to replace the Python virtual machine with an LLVM-based JIT.”
• Performance – When do I start worrying? – “A common problem of the application designers is to predict when they need to start worrying about the Architectural/System improvements on their application. Do I need to add more resources? If yes, then how long before I am compelled to do so? The question is not only when but also what.”
• Upgrading Twitter without service disruptions – “For every incoming tweet, the messaging system gets notifications for all the folowers, which are then processed asynchronously. One of the most important changes they introduced to improve performance in the last nine months is moving from a Ruby messaging middleware to a custom build JVM-based messaging middleware written in Scala.”

Process and Methodology

• Developers should not be allowed to work overtime – “I have a very simple tip for software companies and engineering managers who want to make their developers happier and more productive (and less likely to produce buggy code). Forbid overtime.”
• The Importance of Wireframing – “Since wireframes take considerably less time to generate than full color comps, you can spend time early on using them to map out all of the pages for a particular site. Even at this early stage you can get a sense of the user experience and therefore spot potential usability problems with your design.”
• Get Wireframing: The All-In-One Guide – “Wireframing is a great tool to incorporate into your projects as it allows for rapid prototyping and helps to pinpoint any potential problems. I personally find it invaluable on projects to have a visual representation of content, hierarchy and layout. Overall it’s an excellent step to incorporate into your project before the design process begins for both you and your clients.”
• Idealistic vs. Practical – “Is Agile idealistic or practical? Agile does embraces a set of principles (ideals) and I’ve seen many of its adherents become drunk with the beauty of it. I’ve been there a few times myself. So yes, it is idealistic in that respect. However, I think a better question is this: Is Agile merely idealistic?”

Usability and Acessibility

• Mega Drop-Down Navigation Menus Work Well – “Big, two-dimensional drop-down panels group navigation options to eliminate scrolling and use typography, icons, and tooltips to explain the user’s choices. “
• White Space: How to Get it ‘Right’ – “Space is important in layout. Space can be created by content – such as text, images, lists, logos etc. – or it can be created by the space in—between content, called negative space, or whitespace. Space can be passive; a by—product of the layout process. Or, it can be active; there for a reason, to guide the user’s eye, or help them make a decision.”

Technical Tips

• Fast polling using C, memcached, nginx and libevent – “In this post I’ll show you how to implement really fast polling using C and libevent, memcached and nginx. The performance of the server is over 2400 request pr. second on a not optimized Mac Book – that’s 144.000 requests pr. minute.”

Misc

• Java as Legacy Language – “We owe a lot to Java, clearly, but the sheer cost of using Java for smaller-scale projects is driving the creation of new, lighter-weight languages and frameworks that offer a better impedance match with agile working styles. Going forward, it simply won’t be economical to use Java for most kinds of development.”
• The Elegance of Imperfection – “Bringing heart to web experiences can be difficult, since websites and applications are fundamentally a construct of logic (via code). While you can’t create a website that functions as a pure expression of wabi-sabi, finding ways to infuse our creations with a hint of wabi-sabi adds a new dimension to our work.”

Related posts:

1. Software Quality Digest – 2009-07-27

One of the first decisions we had to make regarding the product was to choose a development stack. This sounds easier than it really is: there are quite a few requirements that a development stack and platform has to meet in order make it suitable for an installable product. When you develop a hosted solution, you don’t have to care so much about things like ease of deployment, cross-platform support or support for multiple databases. Other requirements such as Unicode support are also a lot easier to implement if you can control every little detail of the run-time environment. So these are our main requirements for an ideal development stack:

• Ease of deployment
  Because our customers have to install the application themselves, it must be very easy to deploy. The application will of course come with a convenient installer to setup a database, configure the initial settings and so on. However, a platform with complex dependencies should be avoided as it would cause endless support headaches and would make it much harder for customers to evaluate our product.

• MVC framework
  We have been using different frameworks and patterns for web development over the years and we strongly believe in using the MVC pattern for building web applications. Our ideal web development stack comes with a solid and clean MVC framework to make it easier to build a maintainable application.

• Cross-platform
  We fully expect that 80%+ of our customers will install our web application on a Windows server (those are still extremely popular for internal application and workgroup servers). However, to make it easier for us to offer hosted trials (or even fully hosted application instances), we also need the application to run on Linux machines. Linux support is also handy for customers with distributed teams who want to host our application on cheap dedicated Linux servers on the net.

• Multiple databases
  Similar to the cross-platform requirement, our development stack of choice should support multiple databases. Many companies already have a preference for a certain database (because it’s already installed, maintained and backuped etc), so having easy-to-use support for multiple databases is a big plus. We initially plan to support MS SQL and MySQL only, but we might add support for additional databases such as PostreSQL in the future.

• Support for Unicode and localization
  In an ideal world, every application would have great Unicode and localization support. In reality, though, many applications (and web frameworks) still have a lot of problems with non-Western characters, languages and locales. To make our application suitable for an international audience, good Unicode and localization support in a web framework is key.

• Safe future and strong community
  Of course, the ideal development stack and platform has a strong and clear future. However, it’s not easy to tell which technologies have a safe future. Open platforms with a strong community certainly have better chances to survive in case the owner or original organization behind it loses interest.

I can already reveal that none of the evaluated platforms and frameworks met all our requirements. Some platforms make it really easy to deploy applications and have excellent Unicode support, but fail the cross-platform requirement at the same time. Other frameworks and stacks provide great cross-platform support, but have really bad support for Unicode.

If you are planning to build a web application yourself, make sure to read the upcoming second part of this series to learn about the pros and cons of the available development stacks and platforms (if you haven’t subscribed to this blog yet, here’s the feed). I will also reveal which development stack we have chosen and explain the reasons behind the decision.

Software Quality and Testing

• Quality is Dead #2: The Quality Creation Myth – “One of the things that makes it hard to talk about quality software is that we first must overcome the dominating myth about quality, which goes like this: The quality of a product is built into it by its development team.”
• Coding: Make it obvious – “One of the lessons that I’ve learned the more projects I work on is that the most important thing to do when coding is to do so in a way that you make life easier for the next person who has to come across that code, be it yourself or one of your team mates. I think the underlying idea is that we need to make things as obvious as possible.”
• Introduction to test-doubles – “As soon as you start unit-testing or test-driving your development, you’ll learn about test-doubles and how they can make your tests lightening-fast sooner or later.”
• Future Creep – You aren’t gonna need it! – “How many times have you added an additional parameter or bit of logic to your code because you thought it could be useful and it’s easier to add whilst your already changing the code? If you are your introducing Future Creep! and should stop doing it.”

Performance and Scalability

• Are Cloud Based Memory Architectures the Next Big Thing? – “The next era is the age of Memory and Cloud which will allow for new players to succeed. The tipping point is soon.”
• Measuring efficiency in the cloud – “What can the world of green energy teach us about the future of cloud computing? How can we measure computing resources in the cloud for efficiency, replacement costs, and cost savings?”
• Use Threads Correctly = Isolation + Asynchronous Messages – “Explicit threads are undisciplined. They need some structure to keep them in line. In this column, we’re going to see what that structure is, as we motivate and illustrate best practices for using threads — techniques that will make our concurrent code easier to write correctly and to reason about with confidence. “

Process and Methodology

• How I Learned to Program Manage an Agile Team after 6 years of Waterfall – “If I could go back in time… 3 things I would tell myself about Agile”
• The Power of a Whiteboard – “I think a whiteboard offers loads of advantages over electronic tools. They’re mainly soft factors, I admit, but I think a whiteboard is hard to beat.”
• My Agile Team: More Code, More Problems – “What I’m going to discuss this time is our approach to trying to fix these critical bugs while maintaining at least a semblance of our Agile nature. It’s hard to do a planning meeting and decide on what to work on when you’ve got new things popping up and old things dragging on.”
• If you cannot do a source pull and compile you are doing it wrong! – “For those of you who may NOT think that pulling and compiling is absolutely necessary let me ask you one thing. What value are you providing your organization if a someone who is new to a project (could be new employee, new team member, or even someone with a new computer) cannot pull and compile your source code in about 5 minutes?”
• Customer Development – The Missing Piece! – “I’ve spent a lot of time making agile development work in startups and it ain’t easy. Necessary but not easy.” and a response by Eric from Lesson Learned
• How to Hold the Daily Scrum – “Scrum is simple and Scrum is hard. The Daily Scrum is simple daily routine to help the team self-organize, focus, and identify and eliminate impediments to progress. How do you conduct the Daily Scrum and how do you know if the Daily Scrum is achieving its purpose? “
• Continuous Integration 101 – “More and more people are working in teams where Continuous Integration is used. Not everyone truly ‘gets’ it though… Below is a quick list of things you, as a member of the team, need to keep in mind.”

Usability and Acessibility

• Accessibility 1.0 – “For those unfamiliar with accessibility issues pertaining to Web page design, consider that many users may be operating in contexts very different from your own”
• Brand logos as home links – “My preferred solution would always be for a proper home link but sometimes that is not always practical. Second best would be to combine the word ‘home’ or ‘homepage’ with the logo.”

Security

• IT Security: Blaming the Victim – “Blaming the victim is common in IT: users are to blame because they don’t patch their systems, choose lousy passwords, fall for phishing attacks, and so on. But, while users are, and will continue to be, a major source of security problems, focusing on them is an unhelpful way to think.”

Technical Tips

• Interactive map of the Linux kernel – “There are a lot of books, however it is still a difficult subject to comprehend. The Interactive map of Linux Kernel gives you a top-down view of the Kernel.”

Misc

• The Positive Legacy of C++ and Java – “Java itself will diminish, just as C++ did, to be used in special cases (or perhaps just to support legacy code, since it doesn’t have the same connection to hardware as C++ does). But the unintentional benefit, the true accidental brilliance of Java is that it has created a very smooth path for its own replacements, even if Java itself has reached the point where it can no longer evolve.”
• Do You Need to Be a Developer to Found a Web Startup? – “So it all begs the question, if you’re not from a tech background do you even have a shot at starting a web startup?”
• DVCS and Bug Tracking – “Industry-wide, there is a trend toward more integration between version control and other stuff like project tracking, wikis, discussion forums, build tracking, etc. [...] While DVCS is one of the more interesting things happening right now, it does represent a setback in this particular area.”
• Best Practices for Designing a Social News Website – “In this article I’ll showcase some of the current top social news sites, will identify trends and patterns in their designs and suggest some best practices to follow when designing such sites.”

Related posts:

1. Software Quality Digest – 2009-07-27

Software Quality and Testing

• Quality is Dead #1: The Hypothesis – “You know it’s dead, too, don’t you? You long ago stopped expecting anything to just work on your desktop, right? Same here.” .. and a response by Uncle Bob
• Which Is More Beautiful Architecture – FP or OOP? – “So, who wins? That’s a good question. If you’re in a world resigned to side effects then OOP might just be for you. Else, Haskell and the lessons learned from the language can be powerful instruments in creating rich and beautiful architectures using pure functional techniques. “
• Anecdote Driven Development, or Why I Don’t Do TDD – “Two words: tee dious (sic).”
• Are short methods actually worse? – “Another way of putting it is: short methods may have more defects per line, but they still have fewer defects overall.”
• What makes a system hard to work with? – “My point here is that I definitely believe that there are certain ‘qualities’ of a system’s design that either hinder or accelerate the progress of the system.”
• Slow tests are a bug – “And slow tests are a problem for at least two reasons: they slow down your development and decrease code quality.”

Usability and Acessibility

• Bargain Basement Usability Testing – “We know it is good to focus on usability. We need only look at Apple and the iPod to know that it provides tangible benefits. However, in reality we often shy away. It is hard to prioritize usability when deadlines are looming and budgets are tight.”
• Kindle 2 Usability Review – “The new version of Kindle, Amazon.com’s dedicated e-book device, recently shipped with an improved display and various other upgrades. It now provides good usability for reading linear fiction (mainly novels), though it’s less usable for other reading tasks.”
• Beautiful Forms – Design, Style, & make it work with PHP & Ajax – “We will go through designing and usability you need to keep in mind, styling your form, spicing it up with some nice javascript effects, validating user’s input and finally getting it to work using PHP and Ajax.”

Performance and Scalability

• The Computer Performance Shell Game – “If you’re interested in performance, the absolute first thing you have to do is determine which of these bottlenecks is currently impeding performance — and eliminate it.”
• Performance Impact of CSS Selectors – “This had me convinced that optimizing CSS selectors was a key step to fast pages.”

Technical Tips

• Understand Normalization Before Denormalizing – “Database denormalization is an area which is frequently misunderstood and gets software developers into trouble. Many developers seem to think that denormalization is the antithesis of normalization and have conflicting goals.”

Process and Methodology

• How to be a program manager – “Having a good program manager is one of the secret formulas to making really great software. And you probably don’t have one on your team, because most teams don’t.”
• Sharpening the Saw – “Sharpening the saw is shorthand for anything you do that isn’t programming, necessarily, but (theoretically) makes you a better programmer.”
• The Bare Minimum – “When it comes to project management, I tend to favor the less-is-more approach. For the most part, I think teams should be self-managed and organized. I’ve never seen project managers actually add value – they merely get in the way and slow teams down.”
• Agile, management tools help small team boost software quality – “Indeed, Comet estimates that these tactics have improved software quality by 25% and time-to-market by 10 to 20%.”
• Improving Software Process – A Letter to Upper Management – “Any process, principle, practice we choose should have quality at its heart. Regardless of what process, principles, practices we adopt they will only be successful if we are disciplined enough to follow them.”

Misc

• Wolfram|Alpha Is Coming! – A new search engine by the makers of Mathematica, launching in May
• How to Succeed at Anything – “The good news: Talent is meaningless. You can be world-class at anything.”

Related posts:

1. Software Quality Digest – 2009-07-27