What is Adobe Flash,Flex and AIR?

I think ever since the internet started becoming a platform for business, people have been working on ways to enhance the limitations of HTML.  One way to do this was to get the user to download and install extensions to the basic browser platform that had the capabilities to run the code used to create regular user interfaces. Adobe introduced the Flash player which was essentially an animation and video platform with some development features (Adobe Flash uses a language called Actionscript) .

As the concept of Rich Internet Applications started getting more traction and mind share with the advent of AJAX based technologies – Adobe introduced the AIR platform and the Adobe Flex framework.

The AIR (Adobe Integrated Runtime) platform is a cross platform runtime on which one can deploy applications built using Adobe Flash, HML, AJAX or Adobe Flex on the desktop. Thus AIR is a means by which the developer of an RIA application can extend their presence on to the computers desktop and become independent of the browser.

The Adobe Flex Framework is a bit like Microsoft XAML or Mozilla XUL in that it is uses an XML based language (called MXML) that you use to describe your user interface. You can use in conjunction Actionscript to create applications Flex is an Adobe framework that leverages the Adobe AIR and Adobe Flash run-times. While the framework itself is open source (it’s not free though), the underlying run-times that it targets are proprietary (though I believe they now support the WebKit HTML engine as well which is an open source run-time). Both AIR and the Flex Framework are freely available, Adobe sells an eclipse based tool called Flex Builder that lets easily build powerful applications using the Flex Framework.

Competitors in this space include Microsoft Silverlight and it’s Mono counterpart Moonlight, Mozilla’s Prism project and Sun’s JavaFX.

A predecessor to all these frameworks you might want to consider is OpenLaszlo. It is open source framework I came across a couple of years back that was already supporting flash and I believe now they support a wide variety of platforms.

Another interesting open source project in this space is Curl which is an MIT project

A Linux for every trade…

While puttering around the internet the other day I came upon this software AptonCD and while going through it – a set of light bulbs went off in my head and I actually had an idea ! I was so excited by this that I thought I’d blog about it

I participate in the Free Software Users Group, Thiruvananthapuram (Trivandrum) and one if it’s main activities is making Linux available to the general populace. As part of this activity a local company Zyxware Technologies (these really cool guys BTW) put together vending machine (Freedom Toaster) which burns various Linux distributions on CD/DVD media. This proved to be a very successful project (broadband is still limited and expensive in India so downloading distributions is quite difficult).

One of my challenges as a programmer has been around setting up the development environment. All programmers have a certain toolkit that they are comfortable with. Depending on their level of  sophistication this can be as simple as a text editor to a full blown IDE…  The challenge for me was finding and setting up equivalent tools in the Linux platform to the ones I used in Windows.

It occurred to me  that the challenge I detailed above would be a common one for anyone that is using a set of software tools for their trade.  It would also be useful to people in other professions (other than programming software) that are switching to Linux from another OS platform like Windows to get a set of equivalent tools in Linux for the ones they use in Windows. In fact there are lists out there that detail Linux equivalents to Windows tools.

So here is my idea – We could leverage AptonCD to create meta-packages that people could simply install over the base Linux distribution. These packages would be prepared separately from the base Debian install (Apt is the Debian package manager) and applied after the distribution is installed.

This is not a new idea – in fact here in Kerala we already have a custom Debian distribution that is targeted for schools – IT@School .  The twist here is that while the IT@School is a custom Debian distribution what I have in mind is more in terms of meta-packages that can be installed over a base Debian distribution like Ubuntu. We can leverage the Freedom Toaster to distribute these packages for people. The packages would be created by professionals in a trade  for other professionals who want to use Linux but are not sure how to get all the tools of their trade on it. These people can simply install the OS distribution and install the meta-package for their trade and voila – they can get to work

Of course there are still challenges – Linux often has several tools for a particular task and there may be version conflicts as well. I have also not accounted for the learning curve in getting used to these tools. Nonetheless, I think this would at least give a head-start to professionals wanting to use Linux as a platform for their trade and drive adoption of Linux.

The ultimate aim IMHO is not to make everyone in the world a super-duper Linux hacker but to make people productive in Linux

Prototype It!

So I saw this post today – it’s by Paul Buchheit a former Googler (he is one of the founders of FriendFeed) and the lead developer of one of my all time favorite software applications – GMail (it was his 20% project at Google) and it’s about the concept of Communicating with code.

Paul writes (in his post) on the concept of  using prototypes  to communicate ideas and concepts. He talks about his work with GMail and how he threw together a prototype in order to show the idea of targeted ads in GMail – targeted ads was not a priority until the prototype showed how useful and interesting it could be. The post ends with a similar exercise that he has done using the Friendfeed API (it’s pretty cool – check it out ) The reason I read this post and decided to blog about it is that it talks about developer communication – a topic that I wrote about in another post .

Communicating ideas through prototypes is a great idea – I have always noticed that people get more excited about something they can play with and try out. In fact this is old news in other industries – the auto industry, for example, spends millions to make concept cars to introduce new ideas to the public and solicit feedback Architects likewise – build scale models of their ideas to present to clients. So why don’t we adopt these ideas ? After all we are always talking about “software architecture” and “software construction” and other civil engineering analogies when we talk of software development

My experience is that when the term “prototype” comes up in software development projects most people are thinking of mock-ups. This is especially true in the web-development shops where there is a separate team of graphics designers creating HTML and image pictures of the user interface while a separate team of developers get to “build” the application from the pictures I think this is a very limiting thing.Prototypes should not be limited to the user experience or to presenting and communicating new ideas. I like to make prototypes of my technical solutions to software problems. For example, if you are trying out this great new idea you had on caching data – write a prototype application – the absolute simplest application you can use to exercise your idea.  This would provide you with feedback on whether your idea is valid as well as show up gotchas or limitations in your design.  The pragmatic programmers in their seminal book – The Pragmatic Programmer refereed to these applications as “Tracer bullets”.

Sometimes the concepts or ideas themselves are large and need a lot of programming to even build the prototype(tracer bullet). In these cases I still believe one must prototype the concept – so the question remains – how do you do this ?
Well one way would be to take the approach Paul took in building the initial GMail prototype – modify some existing code. I find sitting in front of a blank file makes the task ahead seem even bigger than it is – so I start with a piece of code to modify even if it is something as simple as a “Hello World” application.  Another good place to look is in the open source forums (though this might be a problem if you are working in a  company that does not allow open source software) – usually there is some variation on your concept that you could work with there . A third option is to accumulate code, links and other resources over time that you can use to jump start your coding.

So – Happy Prototyping

Coding, Puzzles and Mathematics

Coders like to solve problems – the high one gets when one finally ‘cracks’ a problem is one that is as powerful as any drug – they are also alpha males – they don’t like to lose… This powerful combination leads to the traditional image of a pale, pasty, overweight  geek in glasses sitting over a computer screen in the wee hours of the night, mumbling to themselves and ingesting caffeine by the pound.

I’m a bit like that (OK – I am not pale and I like to work in the morning but the rest is similar ) except the problem I might be solving may not even have any real world application. It might simply be a solution to a puzzle !!

This post is about coders and their love of puzzles… Most of us have heard and even experienced the classic puzzle question in interviews for development positions. There are numerous books on this topic (the classic “How Would You Move Mount Fuji? Microsoft’s Cult of Puzzle‘ is a great read) – both about the interviewing techniques as well as methods to solve them. I don’t want to get into a discussion about how good or bad the use of this technique is in interviews (I personally don’t favor them – in the wrong hands they can be horrible – like this one )

What I like to do is examine the puzzles themselves – puzzles can be of several varieties – one can spend hours and hours over them – indeed some people make it a full-time hobby. There are several puzzle competitions  – one of the most challenging being the MIT Puzzle Hunt (the wikipedia entry is very informative). You also have organization leveraging this interest by posting challenges like the FBI

However, the puzzles that interest me are the ones that have a basis in pure logic. I tend to regard puzzles with an eye to the elegance of the solution. A good logical puzzle with an elegantly simple solution is a thing of beauty – I like to go through the reasoning and try and look for extensions or extrapolations. I look at the assumptions and limitations and try to understand the thought process behind the puzzle.

They are a good way to learn, to get ideas on new approaches.  They expand you mind and  are finally but most importantly – they are FUN. I often use them as a fun way to learn new computer languages – good candidates are mathematical puzzles

Here are some nice links – check them out and happy solving

1. Maths Challenges – Check out the links section where there are some programming challenges.
2. Project Euler – Awesome site with some really cool puzzles.
3. Delphi For Fun – The programs are in Delphi which is aversion of Pascal but it has some nice algorithms.
4. FBI Ciphers – Ciphers are a fun type of puzzle that kind off go off in to crypt-analysis.
5. Shine’s Take on IT – My friend Shine is reading “How would you move Mount Fuji?” and is blogging about his read.

Continuous Integration – more than just automated builds

This post is derived from an article by Martin Fowler on Continuous Integration. It my take on Continuous Integration and it’s use.

‘Continuous Integration‘ is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily – leading to multiple integrations per day. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.

Software development is full of best practices that are talked about at great length and detail but rarely actually done. Continuous integration is one such best practice that has been around under various guises for a long time. Anywhere you have heard or discussed the virtue of having a common build that is done in a regular manner – you are hearing about continuous integration. In fact Microsoft has been doing automated daily builds for years.

The term ‘Continuous Integration (CI)‘  is one that originates in the Agile Programming discipline known as ‘Extreme Programming (XP)‘ as one of its twelve original practices. I believe CI has enough merit that it should be implemented even in projects that are not following XP or other agile development practices.

So how does one go about achieving CI within your software development process?
There are two critical aspects to this effort – evangelizing, educating and mentoring the team in CI practices and creating a repeatable process that allows the team to integrate their code quickly, preferably in an automated fashion.

One of the hardest things to express about CI is how it makes a fundamental shift to the whole development pattern; one that isn’t easy to see if you’ve never worked in an environment that practices it. For many people team development just comes with certain problems that are part of the territory. CI reduces or eliminates these problems, in exchange for a certain amount of discipline. The fundamental benefit of CI is that it removes sessions where people spend time hunting bugs where one person’s work has stepped on someone else’s work without either person realizing what happened. These bugs are hard to find because the problem isn’t in one person’s area, it is in the interaction between two pieces of work. This problem is exacerbated by time. Often integration bugs can be inserted weeks or months before they first manifest themselves. As a result they take a lot of finding. With CI the vast majority of such bugs manifest themselves the same day they were introduced. Furthermore it’s immediately obvious where at least half of the interaction lies. This greatly reduces the scope of the search for the bug. And if you can’t find the bug, you can avoid putting the offending code into the product, so the worst that happens is that you don’t add the feature that also adds the bug. (Of course you may want the feature more than you hate the bug, but at least this way it’s you have the choice.)

Though CI is a practice and so does not need any particular tooling to deploy, it is generally found that automation is a key factor in order to make CI. In order to automate the CI process generally it is recommended that the following is done –

1. Keep a single place where all the source code lives and where anyone can obtain the current sources from (and previous versions)
2. Automate the build process so that anyone can use a single command to build the system from the sources
3. Automate the testing so that you can run a good suite of tests on the system at any time with a single command. (The build does a self test)
4. Make sure anyone can get a current executable which you are confident is the best executable so far. (The build creates a deployable version of the software)

All of this takes a certain amount of discipline and consistency in the approach. It is difficult to introduce it in a project but once introduced it does not take that much effort to keep it up. So let us examine the points given above in turn.

1. Keep a single place where all the source code lives and where anyone can obtain the current sources from (and previous versions)
   This is a basic requirement in the software development. Regardless of the size of the team or the complexity of the project all the source code must be available in a single easily accessible location. The versioning requirement means that one must invest in a proper source control system. All the artifacts needed to run the application should, ideally, be located in the source control repository. This would allow the build process to obtain all the resources needed to build the project from a single location.
2. Automate the build process so that anyone can use a single command to build the system from the sources
   Automated builds are essential to create a repeatable process that can be run on demand. This prevents the occurrence of human error in typically complex systems where in addition to building the source code one must set configuration value, etc in order to produce a deployable application. For complicated projects building the source code itself can be a complicated process involving multiple projects and dependencies. Most major software platforms have build tools that help make these tasks simple.
3. Automate the testing so that you can run a good suite of tests on the system at any time with a single command. – make your build self testing.
   Traditionally building involves compiling, linking, etc – all the stuff required to make your program run. However simply because your program loads does not mean that it is working correctly. One very effective way to catch bugs is to run a test suite against your program. These test suites should be built so that they can run automatically against the latest build of the software.
4. Make sure anyone can get a current executable which you are confident is the best executable so far.
   This can be achieved by getting everyone to commit their code to the build often (every day) and ensuring that these commits trigger an automated build every time. This helps ensure that anyone can get the latest workable version of the code at all times – one that passes all the tests. This allows one to be more confident about the code that one is developing will work when merged back in. It also allows testing of the code to be done on the latest version of the code ensuring that bugs being tested are not due to version conflicts or integration issues. It helps testers focus on the changes to the code and identify issues faster as a result. Overall the confidence in the quality of the code is improved.

While CI is a great process – it is like any process only as good as its implementation. There is a abundance of tools out there that do CI or have it as part of their feature set. I will list some that I have dabbled in and heard nice things about -

• CruiseControl .NET This is a open source tool from ThoughtsWorks .
• CruiseControl This is the original java version of CruiseControl.NET
• CI Factory Based in CruiseControl.NET but with a lot of the other agile tools also integrated as well as templates on how to set up your repository, etc
• Hudson Java tool I have seen nice things written about it’s simplicity.

A good book to read is Continuous Integration: Improving Software Quality and Reducing Risk

Update: A Stack Overflow user was asking about Continuous Integration tools today and I pointed him here – I found another useful link while going through all the answers.

ThoughtsWorks have created a feature matrix for CI tools that seems to have all of the major tools covered – link

Programmer – Express thyself

I was in one of my pondering moods the other day and it struck me that software development has less to do with pure mathematical prowess or engineering skill (though having that is certainly a benefit) and more to do with expressing yourself. This is no coincidence – it is because programming software is nothing but an exercise of making ones wishes understood. Indulge me for a minute here while I put to you my thought process…

Ever since we invented a computer, we have been evolving ever more sophisticated ways of communicating with it. As the computer grew more powerful and capable of understanding more complex thoughts we moved from machine code, to assembly instructions to software languages. We have reached a place now that computers are capable of processing huge amounts of data and instructions, executing extremely complex instructions and passing information between each other as needed. In response to these capabilities computers have been programmed using ever larger programs to solve ever bigger problems.

Programming solutions to today’s problems have gone beyond the capabilities of a single individual. Most software projects today require teams of programmers working in concert. This has lead to an interesting dynamic – developing software in groups. Except, software development has moved beyond geography – the ephemeral nature of software makes it an ideal candidate to be developed in a virtual construct that transcends physical proximity. Almost every non-trivial software program developed today is a collaboration of individuals, many of whom are literally at opposite ends of the earth and may have never met face to face.

This leads to a big problem – software programming is a pretty creative and complex process. Add to that, the fact that your collaborators are probably sleeping while you are coding and vice-versa, pile on the disparities in culture and language and you can see why I am saying that software development is more about expressing oneself. One of the bonuses we have being programmers is that, despite all these differences, we are trying to talk to computers and computers have standard ways of communicating with people. So, it can be argued that as long as we all are coding in the same programming language, and we are using the same tools we should be fine. Unfortunately, coding is a only a part of a software project – requirements, design, marketing, users, clients, quality assurance, maintenance, provisioning, etc – all are factors to be considered. All these aspects may be handled by people who are not programmers but still need to know what is going on. Besides, coding itself is much more than simply writing code, it also involves commenting, code reviews, design decisions, trade-offs and alternative approaches, all of which need to be discussed and communicated.

Given all the interactions that I described above, it is no surprise that programmers spend a significant amount of their time talking and writing about the stuff that are doing and are planning to do. Given this dynamic, in my experience, I have noticed that, successful programmers have learnt to express themselves clearly and concisely. I have seen this so consistently, that I would even go so far as to state that, a programmer that cannot clearly articulate a problem either in writing or in conversation has either not understood the problem or is incapable of solving the problem.

Expressing a thought clearly and concisely is a skill that must be practiced. Personally, I am using this blog as much to exercise this skill as I am for anything else. I would encourage you, dear reader, to give thought to this as well. Spend a few minutes over your next email or your next requirements document, proof reading it and making sure you have conveyed you point clearly and succinctly. Reading is another great way to improve your communication skills – look for books outside your normal fare, look at newspaper articles – I have found them to be great sources, since journalist have specific windows of reader-attention and space to convey their points.

Well, until next time… Programmer – Express thyself !!

Case sensitive matches in MySQL

This is an interesting tidbit that I came across in my aimless wanderings around on the Internet. Unfortunately I don’t remember the place that I found it – it was during a random Google search that I had made buried in the second page somewhere.

So the requirement was to do a case sensitive search for values in a MySQL table. So if you wanted to search for records that contained the term ‘foo’ in the title from a table ‘FooName’ containing the following records -

FooName

Title	Description
foobar	a bar of foo
Foo	royal foo
Fooka	car of foo
foo	just foo

The typical select query -

Select Title from FooName where Title like '%foo%'

Returns

foobar
Foo
Fooka
foo

This may not be exactly what you wanted since ‘foo’ is all in lowercase and the query above returns all values that have the letters ‘foo’ in that order regardless of case. In order to ensure case sensitivity in MySQL just modify your query like so -

Select Title from FooName where binary Title like '%foo%'

Returns

foobar
foo

MySQL Resources

One of the things I have discovered the hard way about working with MySQL is that unlike SQL Server 2005 resources for MySQL are not as widespread. Other than the official MySQL website and the MySQL sponsored forums there seem to be a strange lack of technical resources on MySQL. Since there seem to be so little I thought it mights be beneficial -

• MySQL Reference – The online link to MySQL 5.0 is here. It is good reference but is a little skimpy on the examples unless you dig into the comments section at the bottom of the pages.
• Xaprb – This is one I am a big fan of – the blog is focussed on MySQL. The author really knows his stuff, not only in MySQL esoterica but about SQL in general. Great place to go to learn more about MySQL.
• MySQL Performance Blog – This is another blog written by MySQL consultants based in Europe. This blog is focussed more on the performance tuning tips for MySQL. Great resource to learn the various settings and variables in MySQL that can be used to diagnose and tweak MySQL performance.
• MySQL Forge – This is MySQL’s version of Sourceforge.com. This is a community site for MySQL tools and scripts. Lots of perl, php script tools dealing with various aspects of database maintenance and functionality.
• MySQL – The definitive guide to using, programming and administering MySQL 4.1 and 5.0 (ISBN: 0672326736) – This book by Paul Dubois is a great reference for MySQL. As befits a definitive guide the book is hefty at nearly 1300 pages but a must have for any in depth study of the database.
• mySQL CheatSheet – There is a neatly laid out cheat sheet in this blog post. I find it useful lying on my desk for the times I blank out on stuff.
• There are a couple of blogs by mySQL insiders that I keep an eye on – http://jan.kneschke.de/projects/mysql and http://www.planetmysql.org/robin/

Thats about it for now. Please feel free to add ones you guys go to in the comments to the post (the comments are moderated so they may not appear as soon as you insert them)

Getting Microsoft SQL Server and mySQL to talk

One of the biggest things I have to do as part of my company’s transition to using mySQL is the migration of the existing data to the mySQL platform. The problem was getting Microsoft’s SQL Server 2005 and mySQL to talk in a straightforward and flexible manner. The main issue is that SQL Server 2005 does not provide ODBC drivers out of the box.

This IMO is a big backward step for SQL Server 2005. If you use the SQL Server Management Studio interface there is .NET ODBC driver available as a data-source when trying to import data but when you try to make ODBC as your data-sink these options are no longer available. You get this issue when using SSIS also so I went on another tack – linked servers.

Linked servers are simply connections to other data sources or data sinks that are maintained by the Microsoft SQL Server. These can be local or remote and are a flexible and powerful way of making data that is spread across multiple servers available. You big advantage is that you could address the data with SQL statements directly as if they were on the same machine. Linked Servers could be ODBC data-sources as long as the appropriate ODBC drivers were available.

So I went to mySQL and got the mySQL ODBC Driver. After installing the driver on the server (mySQL has detailed installation instructions here), setting up the linked server was pretty straightforward -


EXEC master.dbo.sp_addlinkedserver @server = N'MYSQL', @srvproduct=N'MySQL', @provider=N'MSDASQL', @provstr=N'DRIVER={MySQL ODBC 3.51 Driver}; SERVER=127.0.0.1; DATABASE=DB_Name; USER=UserName; PASSWORD=Password; OPTION=3'


Now you have set up a linked mySQL server using ODBC. Since this is an ODBC connection selecting and inserting data to and from it is slightly different. You use the openquery function like so -

Selecting Data -
select * from openquery(MySQL, 'select * from `db_name`.`table_name`')
Inserting Data -
INSERT into openquery(MySQL,'select * from db_name.table_name')
select * from db_name.dbo.table_name


mySQL Tools

Coming from a cocooned world of Microsoft development where the SQL Server Management Studio pretty much handled the spectrum of database administration and management – my immediate focus was to find similar tools for my SQL.
The surprising thing I found was that while GUI tools on Windows for mySQL were plentiful – GUI tools for mySQL on the Linux environment were scarce. On further reflection the assumption that there would be GUI tools on Linux shows considerable naiveté’ on my part. After all, if you are using the Linux environment on a day to day basis you would probably be looking for command line and shell scripting tools rather than GUI tools.
mySQL has adopted the approach of exposing an api for 3rd party tool builders. They have built a set of pretty basic tools that I talked about earlier but nothing that performed from end to end.
Once I let go of the approach of looking for 1 tool that did everything, however, I found better success. While still meager on the Linux side – there are some pretty useful tools for mySQL in the windows world .
My picks for desktop based tools are :

1. SQLYOG :- This tool is the product of a small Indian software company. However it has a decent Query interface, data migration tools as well as good schema management tools. The price is a perfectly affordable $70 for the enterprise version. The only quibble I have with this tool is that it seems to use up a lot of CPU and memory.
2. Toad for mySQL :- TOAD doesn’t need much introduction to database administrators. It is pretty much the de-facto tool that is used for Oracle. They have built a free-ware version of their tool which is pretty compelling. Admittedly it is lighter on features and does not seem to be able to browse data without throwing up errors but this would be a good solution to those that are used to TOAD and thus can avoid too large a learning curve. And last but not the least – its freeware.
3. DeZign for Databases :- This tool was recommended by a colleague (thanks Scott) as great database modeling tool. The trial version is hobbled in that all table are limited to a maximum of 8 columns but the tool IMO is well worth the investment. This is especially true if you need to keep updated ER Diagrams of all your databases and synchronize both from and to the database is needed.

There are a few web-based tools worth looking into also :

1. PHP MyAdmin :- This is the one that is touted by most everyone who has worked with mySQL. It is cross-platform, easy to set up on central location, easily accessed by anyone with an internet connection and is open source. The main complaint I have with this tool is that it is web based and therefore cannot handle tasks that require significant amounts of time. This effectively eliminates it IMO as a tool that can be used for an enterprise level database with significantly large amounts of data. It is a great tool, however, for small databases that require a limited amount of tuning. I also find the interface a bit confusing and a little dated – but thats mostly the web developer in me talking
2. MonYog :- To be honest I haven’t really used this this tool very much. It is a beta product distributed freely by the makers of SQLYOG. It is an AJAX enabled tool used for monitoring the mySQL database.