Open Source Strategies

This blog is moving!

2009-06-23T10:20:00.000-07:00

After four years, this blog has a new home. We've just upgraded www.opensourcestrategies.com to use WordPress. We hope you'll continue to follow opentaps Open Source ERP + CRM and our life as an open source software company there.

New screens for creating purchase orders in opentaps

2009-05-12T15:19:00.001-07:00

If you have been using opentaps 0.9 or 1.0, you should find the new screens for creating purchase orders in the upcoming version 1.4 much easier to use.

First, the screen for initializing purchase orders has been streamlined. Instead of having to go through two screens to select a supplier and agreements, there is now one screen to do both:Second, when you're adding items to your purchase order, there is now an auto completion which it to prompt you for a product based on either the product ID or a portion of the product name

If the product has not been set up already with a supplier entry in the catalog manager, opentaps 1.4 will let you still add it to your order:
Finally, to create your purchase order, there is now just one screen for setting the terms of your purchase order and selecting a shipping destination:

opentaps Open Source ERP + CRM Quarterly Update

2009-04-29T14:13:00.000-07:00

Opentaps Now Supports Hibernate

We made a very significant and fundamental enhancements: opentaps now supports hibernate as well as the original ofbiz entity engine. You can use the two interchangeably to store and retrieve data, but the addition of hibernate also gives us a more flexible query language, data validation, search, and distributed database capabilities. We think this will open up a lot of new opportunities for opentaps down the road.

Asterisk VOIP PBX Also Integrated

Another important enhancement is the integration of opentaps with the asterisk open source voice over IP (VOIP) PBX. You can now use it with opentaps CRM, purchasing, and other applications for inbound and outbound calls.

Enhanced Manufacturing and Inventory Planning

The inventory and manufacturing planning capabilities of opentaps, one of our traditional areas of strength, were enhanced with several new features, including:

Support for routing specific Bills Of Material (BOMs)
Minimum and maximum quantities for manufacturing routings in Material Resources Planning (MRP)
Ability to edit and consolidate manufacturing and purchasing requirements
Trace the source and usage of inventory items through their entire lifecycle
Allow orders to be reserved in and shipped from multiple warehouses

An Easier Way to Segment Financial Data

Many organizations would like to be able to segment their financial results, and we introduced a new accounting tags feature which allows you to add up to 10 tags to all your invoices, payments, and general ledger transactions. For example, you can tag your transactions by division, department, activity, and cost center, and then see standard financial reports such as income statements, balance sheets, and cash flow statements for the different combinations of tags.

More Automated Testing Resources

We now have over 350 automated tests for the opentaps development trunk, which are run on both MySQL and PostgreSQL daily. We have also integrated the Selenium and Fitnesse testing frameworks into opentaps, so that we can add more automated front end testing as well.

The World of Opentaps

It's been almost 3 years since we unveiled opentaps at the MySQL Users' Conference in Santa Clara. Since that time, opentaps has grown both in capabilities and in its community. Last month, I used google map to make a map of where some of the organizations which use or provide services for opentaps are located, and the result is this map -- the World of Opentaps:

Take a look -- there's probably somebody near you!

The High Cost of ERP and the open source alternative

2009-04-27T11:47:00.000-07:00

CFO research published a white paper last month about the high cost of ERP, and the results were pretty astounding: ERP software is even more expensive than most people thought! In their survey, over 80% of the companies had customized their ERP systems, and the annual cost of making those customizations can be nearly twice the amount of annual maintenance and support fees. This is because most commercial ERP systems are very difficult to change. Even minor modifications took on average 7 person-days to make.

As a result of this high cost of customizing ERP software, many CFO's are giving up completely on customizations and instead, as one finance director said, "... will modify the business process if necessary or create an offline procedure" instead of making customizations. In other words, either they will run their business as their ERP vendor tells them, or they won't use the ERP system at all and go back to the "offline procedures" of custom spreadsheets, stacks of paper, and whatever else our grandparents used, before computers became popular.

Allow me to point out the obvious: customizations are inevitable, because each business is unique, and also because the business environment changes. If your ERP software cannot accommodate that, then it's simply the wrong solution. Adapting to it is not following "best practices" -- it's more like putting yourself into a straitjacket.

Forunately, open source ERP solutions, such as our opentaps Open Source ERP + CRM, are fundamentally easier to customize because of the following reasons:

You have access to the source code, so instead of having to follow painful vendor workarounds, you can modify the source code to fit your needs.
opentaps is newer than most of the commercial ERP software and architected for change: it is more modular, more object-oriented with our new domain driven architecture, and supported by a large body of automated tests.
opentaps is designed for a wider audience, and hence intended to be customized to fit a much larger range of uses. Take a look at the world of opentaps, and you'll see how organizations all around the world have tailored opentaps to their needs.

The World of Opentaps

2009-04-16T17:45:00.000-07:00

I finally figured out how to use Google Maps and made this map of where some of the different users and service providers of opentaps are located:

There are probably quite a few that I've missed, but this should give you the general idea. My how we've grown!

opentaps integrated with Asterisk

2009-03-19T11:32:00.000-07:00

We've just finished a feature that a lot of you have been asking about: the integration of opentaps Open Source ERP + CRM and the Asterisk voice over IP (VOIP) PBX.

With this integration, you'll be able to dial out from opentaps using Asterisk. When you get an inbound call, Asterisk will tell opentaps, which would then show this little widget (made with the Google Web toolkit) that can take you to the page with information about the caller:

This feature will be part of our upcoming version 1.4 release for opentaps. In the meantime, you can take a look at our tutorial on integrating opentaps with Asterisk and try it out -- it's already part of our development repository.

By the way, if you've never tried Asterisk, you can set it up pretty easily on Amazon EC2 and try it yourself. We have some information about that in our tutorial as well.

Open Source in a Wider World

2009-02-16T13:45:00.000-08:00

Last week, I wrote about how the European Commission's Tool East project leveraged opentaps Open Source ERP + CRM to create an open source ERP system for the Eastern European tool and die making industry. I thought this was a very interesting example of how open source software could be used to advance social and economic development.

I was pleasantly surprised to hear encouraging feedback from many people about this project. In a time where our environment and our economies are facing unprecedented challenges, it's gratifying to know that our work could help our societies meet those challenges. I hope that the open source communities could come together and solve greater social problems in the future.

opentaps Open Source ERP + CRM Quarterly Update

2009-01-30T14:12:00.000-08:00

In the last few months, we've introduced a few interesting new features to opentaps.

Query Builder for opentaps Analytics

We have completed a query builder for opentaps analytics, which allows you to query the opentaps data warehouse based on multiple criteria. For example, you can now run a report of sales by product for all "Gizmos" in the United States since 2007, or sales by product in California on Mondays during the Julys. This new tool, along with the data transformations, reports, and report generator we've already built, will give you a large number of available reports through opentaps analytics

Google Web Toolkit Widgets Now Officially in opentaps

We have merged back the Google Web Toolkit (GWT) widgets into the main development trunk of opentaps. These widgets will be part of the upcoming version 1.4 release. They will allow you to add and search for data much more quickly than before, while still allowing you to export lists of data to Excel.

opentaps - Asterisk Integration Under Way

We have also started developing an integration between opentaps and the Asterisk open source voice over IP (VOIP) system. This integration will also allow you to call out on your Asterisk VOIP phone from opentaps. When you receive a call on your Asterisk phone, a GWT widget will pop up with information about the caller and take you to the screen in opentaps with their information.

Other Development of Note

Some other important developments during the last few months include:

Improved commission management and reporting. Commissions can now be issued when the customer invoices are paid, instead of just when the customers are invoiced. There is also a new commissions report to show the sales, payments, and commissions in detail.
New cash lock box management feature, which allows you to import NACHA files from your bank to create payments automatically in opentaps.
Material Resources Planning (MRP) can now account for sales forecasts as well as actual sales, purchasing, and manufacturing.
opentaps 1.0.3 was released as an upgrade for users of opentaps 1.0.x

Upcoming Release Plans

The next major release of opentaps will be version 1.4, which we are currently planning for Summer 2009. Between now and then, we will be using the new GWT tools to improve the user interface, especially order and invoice entry screens. We will also be making some more fundamental improvements to the opentaps technical infrastructure.

opentaps Quarterly Update

2008-10-23T17:28:00.000-07:00

During the last few months, we've continue to improve the technical foundation of opentaps in two important directions. In the backend, we've migrated key portions of our customer, order, inventory, and general ledger code to the new domain driven architecture. In the process, we've been able to make the underlying code more flexible and easier to maintain and improve its performance. (See this example.)

In the front end, we have started to integrate the Google Web Toolkit into opentaps to build more friendly user interfaces. This toolkit will allow us to turn our web forms into interactive widgets that look and feel like a more responsive desktop application. For example, take a look at this Flash demo, and you will see a widget which acts as its own application. It gets data from opentaps, checks user input, and connects to the server without affecting the rest of your screen. The lists of results are also more dynamic: you can sort them, show or hide fields, and rearrange the layout. The interface is also a lot faster and smoother.

Some other recent enhancements include:

Ability for migrating data from NetSuite
Support for tracking inventory at standard costs
Better support for orders with multiple shipping destinations
Additional reports and enhanced user interfaces in opentaps analytics
Upgrade of opentaps analytics to Pentaho 1.7
opentaps 1.0.2 released.

Looking ahead

In the next few months, our key initiatives will be integrating the Google Web Toolkit more smoothly into opentaps to make building user-friendly web applications as easy as possible for you. Separately, we will also be looking to improve on the business execution, or services, layer of opentaps. Our goal is to provide you with better support for job scheduling, workflow management, and remote Web services.

Cleaner and Faster Code

2008-09-26T17:28:00.001-07:00

Counting inventory should be pretty straightforward, but it doesn't have to be. In early versions of opentaps, it was handled by this block of minilang (an XML scripting language from the ofbiz framework) from the ofbiz product application:


       
        <set from-field="parameters.productId" field="lookupFieldMap.productId"/>
        

        <if-compare field-name="parameters.locationSeqId" operator="equals" value="nullField">
            <set from-field="nullField" field="lookupFieldMap.locationSeqId"/>
        </if-compare>

        <if-compare field-name="parameters.useCache" operator="equals" value="true" type="Boolean">
            <find-by-and entity-name="InventoryItem" map-name="lookupFieldMap" list-name="inventoryItems" use-iterator="true" use-cache="true"/>
            <else>
                <find-by-and entity-name="InventoryItem" map-name="lookupFieldMap" list-name="inventoryItems" use-iterator="true" use-cache="false"/>
            </else>
        </if-compare>
        
        <set field="parameters.availableToPromiseTotal" value="0" type="Double"/>
        <set field="parameters.quantityOnHandTotal" value="0" type="Double"/>
        <iterate entry-name="inventoryItem" list-name="inventoryItems">
            <if-compare field-name="inventoryItem.inventoryItemTypeId" operator="equals" value="SERIALIZED_INV_ITEM">
                <if>
                    <condition>
                        <or>
                            <if-compare field-name="inventoryItem.statusId" value="INV_AVAILABLE" operator="equals"/>
                            <if-compare field-name="inventoryItem.statusId" value="INV_PROMISED" operator="equals"/>
                            <if-compare field-name="inventoryItem.statusId" value="INV_BEING_TRANSFERED" operator="equals"/>
                        </or>
                    </condition>
                    <then>
                        <calculate field-name="parameters.quantityOnHandTotal" type="Double">
                            <calcop field-name="parameters.quantityOnHandTotal" operator="add"><number value="1.0"/></calcop>
                        </calculate>
                    </then>
                </if>
                <if-compare value="INV_AVAILABLE" operator="equals" field-name="inventoryItem.statusId">
                    <calculate field-name="parameters.availableToPromiseTotal" type="Double">
                        <calcop field-name="parameters.availableToPromiseTotal" operator="add"><number value="1.0"/></calcop>
                    </calculate>
                </if-compare>
            </if-compare>
            <if-compare field-name="inventoryItem.inventoryItemTypeId" operator="equals" value="NON_SERIAL_INV_ITEM">
                <if-not-empty field-name="inventoryItem.quantityOnHandTotal">
                    <calculate field-name="parameters.quantityOnHandTotal" type="Double">
                        <calcop operator="get" field-name="parameters.quantityOnHandTotal"/>
                        <calcop operator="get" field-name="inventoryItem.quantityOnHandTotal"/>
                    </calculate>
                </if-not-empty>
                <if-not-empty field-name="inventoryItem.availableToPromiseTotal">
                    <calculate field-name="parameters.availableToPromiseTotal" type="Double">
                        <calcop operator="get" field-name="parameters.availableToPromiseTotal"/>
                        <calcop operator="get" field-name="inventoryItem.availableToPromiseTotal"/>
                    </calculate>
                </if-not-empty>
            </if-compare>
        </iterate>

        <field-to-result field-name="availableToPromiseTotal" map-name="parameters"/>
        <field-to-result field-name="quantityOnHandTotal" map-name="parameters"/><

If you can't figure out what this does, you're not the only one. Read on. (I told you counting inventory should be pretty straightforward, remember?)

As part of our restructuring toward a domain driven framework, we decided to clean it up. First, the task of retrieving the list of eligible inventory items was pushed to the InventoryRepository:



 public List<InventoryItem> getInventoryItemsForProductId(String productId) throws RepositoryException {
        try {
            List<GenericValue> inventoryItems = getDelegator().findByAnd("InventoryItem", UtilMisc.toMap("productId", productId));
            return (List<InventoryItem>) Repository.loadFromGeneric(InventoryItem.class, inventoryItems, this);
        } catch (GenericEntityException ex) {
            throw new RepositoryException(ex);
        }
    }

Then, we felt that whether an item was serialized or not, and calculating the net inventory quantities of that item were rightfully properties of the inventory item itself, so we made them part of the InventoryItem class:


    public Boolean isSerialized() {
        return "SERIALIZED_INV_ITEM".equals(this.getInventoryItemTypeId());
    }

    public Boolean isOnHand() {
        String statusId = this.getStatusId();
        return "INV_AVAILABLE".equals(statusId) || "INV_PROMISED".equals(statusId) || "INV_BEING_TRANSFERED".equals(statusId);
    }

    public Boolean isAvailableToPromise() {
        return "INV_AVAILABLE".equals(this.getStatusId());
    }

    public BigDecimal getNetQOH() {
        if (isSerialized()) {
            if (isOnHand()) {
                return BigDecimal.ONE;
            }
        } else {
            BigDecimal qty = getQuantityOnHandTotal();
            if (qty != null) {
                return qty;
            }
        }
        return BigDecimal.ZERO;
    }

    public BigDecimal getNetATP() {
        if (isSerialized()) {
            if (isAvailableToPromise()) {
                return BigDecimal.ONE;
            }
        } else {
            BigDecimal qty = getAvailableToPromiseTotal();
            if (qty != null) {
                return qty;
            }
        }
        return BigDecimal.ZERO;
    }

As a result, counting inventory was as simple as this:


 public void getProductInventoryAvailable() throws ServiceException {
        try {
            InventoryDomainInterface inventoryDomain = getDomainsDirectory().getInventoryDomain();
            InventoryRepositoryInterface inventoryRepository = inventoryDomain.getInventoryRepository();
            
            List<InventoryItem> items = inventoryRepository.getInventoryItemsForProductId(productId);
            
            availableToPromiseTotal = BigDecimal.ZERO;
            quantityOnHandTotal = BigDecimal.ZERO;

            for (InventoryItem item : items) {
                availableToPromiseTotal = availableToPromiseTotal.add(item.getNetATP());
                quantityOnHandTotal = quantityOnHandTotal.add(item.getNetQOH());
            }

        } catch (GeneralException ex) {
            throw new ServiceException(ex) ;
        }
    }

It really is as simple as the code reads -- get a list of the inventory items and out of their quantities. (Go back to the top and read the minilang XML again, and you'll see that's what it does as well.)

But it gets better: by switching from minilang to object-oriented Java, we got not only cleaner code, but a performance improvement of about 25% as well.

first pass at a domain driven architecture

2008-08-18T16:11:00.000-07:00

We've completed the first pass of the new domain driven architecture, so I wanted to give you an update about how it works and tell you about some of the interesting things we found out along the way.

How It Works

The basic idea of the domain driven architecture is to separate your application's business logic from the underlying nuts and bolts, so that you can use several different applications interchangeably or migrate from one platform to another while preserving the essential core of your application. Another related goal is for you to be able to write business logic code in business terms, so instead of working with tables in a database or their equivalents, you can work with business concepts such as customers, orders, and invoices. Later, when someone else comes in to read your code, it will be easier for them to understand what you've done and help maintain it going forward.

To do this, we have implemented a set of entity class objects representing the data model of opentaps. Most of these classes are automatically generated from the original entitymodel XML's, and you can extend or reuse them to implement more complex business logic. The actual interactions with external resources such as databases or legacy services are defined as Java interfaces, and a central domains directory defines where they are implemented. So, you can define an OrderRepositoryInterface of the methods for obtaining order related information, and your Order domain can interact with this interface. The actual implementation of it is in the OrderRepository, and at run time opentaps will use Spring to look in the domains-directory.xml file to find the order domain that you are actually using and get the order repository from it. If later you decide to switch from the opentaps order management system to another order management system, you can just re-implement all the Order domain interfaces, and the core order processing logic will continue to function.

What We Learned

A couple of the things that we realized as we work on defining the domain driven architecture:

We decided it was necessary to implement a true object layer for the opentaps data model. I originally thought that this was unnecessary, and we could just extend the old ofbiz GenericValue into our domain objects as needed. This turned out not to work, though, because we'd be stuck with chunky GenericValue objects instead of lightweight Java objects. More importantly, it would have introduced logical inconsistencies. For example, imagine if you implemented a method called getTotal(), and later somebody introduced a field called "total" to an entity. If you were extending GenericValues to Java objects, get("total") and getTotal() would give you different results.

My original plan also called for an Infrastructure class which provided the "plumbing" for each platform. Thus, for the ofbiz-based applications there would be the delegator and dispatcher, and for a hibernate-based application there will be a session factory, etc. etc. This actually made the code very messy, because we could not define a common Infrastructure interface which could be used across domains. Each domain with have to use an Infrastructure which is specific to its framework. After further thought, though, I realized that what we really needed was a common Infrastructure that was global to all the frameworks used in opentaps. This did not compromise the reusability of any particular domain implementation, and has a nice benefit of being able to transfer resources from one platform to another.

The Next Step

Now that we have the structure in place, we will be reimplementing some of the existing features with the new domain driven architecture and in the process refine it further. The initial emphasis will be on parts of the application that could really use an object-oriented redesign, either to improve code manageability, like the Party and contact information features, which could benefit from inheritance, or performance, by rewriting some old minilang XML code in Java.

opentaps Quarterly Update

2008-07-15T18:30:00.001-07:00

We've been busy! Now that we're past version 1.0, we had a chance to make some fundamental improvements to opentaps for a long-term. We have begun to develop a new domain driven architecture for future versions of opentaps. This object-oriented architecture will make it easier for you to customize and extend opentaps or to combine it with other applications. It will also make it easier for us to develop opentaps and help it continue to grow.

opentaps Analytics is also coming along nicely. You are now able to slice and dice customer, order, and return data by over a dozen different ways, including by country, by brand, by month of the year, and by category. You're also able to look at the lifetime value of your customers from all these different angles. We are completing development of an automated query generator for JasperReports which will allow us to create a lot of reports quickly and easily. (See the screenshots.)

In addition, we've made the following improvements:

A maintenance release of opentaps 1.0.1
A new query tool to make querying and reporting easier to develop
A POJO (Plain Old Java Object) Service Engine for mounting business logic written in standard Java objects (beans) in the service engine.
Support for order management and invoicing of services
Support for kerberos single sign-on
Improvements to data importing, order entry, manufacturing, and warehouse management
And last but not least, thanks for a community contributors, a translation of opentaps to Bulgarian and new documentation pages for opentaps in Spanish and Chinese.

Coming Soon

These are some of the additional enhancements planned in the next few months:

Ability to deploy opentaps on JBoss
Support for LDAP authentication
Better Ajax look up interfaces
Improved marketing features, including customer segmentation and Recency, Frequency, Monetary (RFM) analysis

A New opentaps

Development of opentaps will continue towards version 1.4 using the current ofbiz-based framework in the next few months. The coding style will become more domain-driven, object-oriented as our domain driven architecture becomes more established. Later this year, we will begin developing a new framework for opentaps 2.0. This new framework will be fully compatible with existing and future applications built on ofbiz, but it will rest upon a stronger Java J2EE foundation and allow us to incorporate many other open source projects into opentaps. This will allow us to expand opentaps into whole new horizons much more easily than ever before. We can't wait!

Finally, a special thanks to Simon Zhao, Juan Gimenez, Ivan Popov and the Tooleast team, and Skip Dever for their contributions to opentaps in the last few months.

The Value of Software Architecture

2008-07-02T09:15:00.000-07:00

Every company, no matter how big or how small, already has software to manage their business. So if they are looking at opentaps Open Source ERP + CRM, it's because their system has failed them. That system may still meet some needs well, but as a whole it can no longer support the users' long-term needs.

Why this is happening? Usually for the following reasons:

The users have no access to the source code, so they have no way to make changes they need.
The system can't support new functionality. For example, it may have been built solely with direct marketers in mind and cannot be adapted for manufacturing or more complex B2B sales.
The source code is no longer maintainable because its original authors have moved on, and there's a lack of documentation and unit tests to help new developers work with it.
The system cannot support new technologies. It may have been written with a DOS-based rapid application development tool, which will never support the Internet.
The hardware which runs the system is no longer made or supported. For an extreme example of this, see this story about Ecometry and the HP3000.

What We Can Do (Better)

These problems show how important software architecture become over a long period of time. The most important thing about well architected software is that it can withstand change. It should allow you to modify, extend, or even replace parts of the software while the rest of it keeps humming along. In this way, a good architecture could extend the lifespan of your software and allow it to adapt to new technologies, new people, or new requirements.

For us, as developers of opentaps, it means that we need to produce a system that is not only feature rich and user friendly, but also a well architected. In this respect, there are somethings we already do well, and there are somethings that we need to do a lot better.

I think we already do the following well:

Our source code is openly available to all of our users, and this is a huge advantage, because it allows them to modify opentaps to meet their needs. In theory, this should actually be all they need, but in practice that is not the case. Having access to the source code doesn't actually do you any good unless the source code is also easy to work with and can support changes easily.
We have a very robust data model, which can support the core business needs of most industries.
Because opentaps is a newer system, it is built with current industry standard technologies such as Java, SQL databases, and XML.

However, there are some things that we need to do a much better job of. Much of our code is still too static and closely tied to the underlying data structure and one particular framework. (See this example.) What we really need is to adopt a more object-oriented design, so that it's easier for other developers to understand and extend opentaps, and so that we can be up-to-date with the best open source technical frameworks available.

Therefore, we have begun implementing a Domain Driven Design for opentaps, which is an object oriented design pattern that builds applications from business domains such as customers, orders, invoices, and shipments and separates the infrastructure tier of dealing with servers and databases from the business knowledge. We believe that this Domain Driven Design will give us -- and most importantly you -- the ability to extend opentaps, replace some parts of it with other systems, or upgrade opentaps to newer technical infrastructure, while preserving your investment in the rest of it.

The Plan for opentaps

This Domain Driven Design will be an evolutionary change from the current architecture. We will be moving our business tier code to the new design over the next few months while continuing to piggyback off the existing ofbiz framework. While doing this, we will continue to advance the development of opentaps version 1.4.

At some point, we will begin creating a new framework for opentaps version 2.0. This will be fully compatible with the ofbiz framework, allowing you (and us) to incorporate current and future versions of ofbiz and all the opentaps and custom applications we have already built with it. The drivers of the new opentaps framework, however, will more likely be a yet to be decided combination of open-source projects such as Hibernate, Spring, guice, JBoss Seam, Struts, Wicket, etc. It will allow us to re-factor, at our leisure, legacy ofbiz-based code to this new framework and at the same time incorporate other Java open source projects into opentaps more easily.

First Look at opentaps Analytics

2008-06-10T09:36:00.000-07:00

Last week, I uploaded the first screenshots for opentaps analytics to the opentaps documentation wiki. While this application is still in its early stages, I thought he screenshots should help you see what we've been working on.

What you're looking at is an application that takes operational data from the opentaps Open Source ERP + CRM system and transforms them into an analytical data warehouse using the Kettle ETL tool. Then, from this data warehouse, we have built dashboards, maps, drill downs, and reports.

The goal of opentaps analytics is to help you make better business decisions with your data by providing you with a standardized out of the box set of analytical tools and reports. At this point, we have the initial technical framework for doing this in place, thanks to MySQL, Pentaho, Mondrian, and JasperReports, and we will be extending it with more business analysis.

Evolution of Enterprise Software

2008-05-21T20:28:00.000-07:00

A couple of months ago, I wrote about the hierarchy of enterprise software needs, and today I would like to share with you how I think those needs have changed.

A long, long time (say ten years) ago, a company might decide that it needs to manage its sales effort more effectively. It might decide that a CRM system is what it needed, then buy servers, license a package, and then train, or at least try to train, its salespeople to use the system. They are told that they need to enter all their account and contact information into the system, log all their calls and meetings with clients, and fill out forms to record the sales opportunities. They will also have to make periodic forecasts with the system.

Needless to say, some sales people might resist, but it's up to management to make sure that they use the system. Some of these measures might be draconian -- I once read about a company who would not pay sales commissions on orders which were not first created as opportunities in its CRM system! If the salespeople ultimately use the system, however, it could be very helpful, because with it management could see what's going on -- how many leads there are in the pipeline, what stage all the sales opportunities are, which customers have gone dormant, etc.

Today, we're in a different world. Thanks to the digital explosion of the past decade, important data such as accounts and contacts, not to mention e-mails and appointments, could be spread out over Blackberries, cellular phones, Facebook, LinkedIn, Google calendars, Gmail, Hotmail, Yahoo Mail, and a thousand other places. While there's still a core organization whose data is maintained in accounting, order entry, and warehouse management systems, data vital to the success of the organization is also increasingly spreading out like a web into other systems and devices.

This means that the enterprise software of the future could no longer meet an organization's needs by solely looking inward to the data that's available inside of the organization. Rather, it will need to fan out into this web of critical data, much like a search engine spider would, and connect with all the important online services and devices used by key employees. It must be able to bring all that information together with data from core systems like accounting, order entry, and warehouse management, to coordinate activities and provide analysis.

This is the next evolution of enterprise software, and it's where we will be taking opentaps Open Source ERP + CRM. We've got a good start already:

We can sync data with mobile devices and online service providers
We have a very robust, normalized, operational data model that could captured the true richness of business relationships and transactions.
Finally, we have some very strong open-source business intelligence tools bundled with opentaps that can make sense of it all.

Our next step is to tie all those pieces together, so that our users could see not only what is tracked by traditional business systems but what is happening in the entire web around their organization, perform detailed analysis, and take actions based on that information.

Social Networking and Ecommerce

2008-05-09T16:59:00.000-07:00

There's an article about social networking and online retailers in this month's Internet Retailer.

Why CRM Fails Many Companies (and Can We Fix It?)

2008-05-02T09:00:00.000-07:00

A study just released by the Chief Marketing Officer Council showed that a large percentage of marketers are unhappy with their CRM systems. Specifically:

45% believe CRM systems are not effective enough
85% do not think they can integrate disparate customer data sources well
94% do not believe they have an excellent knowledge of customer demographics or transaction histories

As a result, even though nearly a third of the companies surveyed have customer churn rates of over 10%, two thirds have no system in place to go after these lost customers. In effect, most companies' marketing efforts resemble a "leaking bucket" -- a lot of effort is spent getting customers in, but not much is done to keep them.

Part of the problem may be cultural. Perhaps we think of "growth" as adding new customers rather than serving existing ones better. A big part of the problem, though, is surely technological. It is harder to figure out how to generate additional revenues from our existing customers than to go after new ones. For example, there's not much involved in renting mailing lists, putting out more advertisements, or paying more for online clicks. But how many companies could really pull together all their transactional data from their retail stores, online stores, and other sales channels, and use that data to figure out what they should be doing?

This is exactly the problem we will try to solve with opentaps analytics, where we're building a set of standard tools to help you better understand your customers. We have just gotten started with this effort and made some good progress, and I will make more posts as we go along.

O'Reilly Web 2.0 Expo

2008-04-28T10:11:00.000-07:00

I was at the O'Reilly Web 2.0 Expo in San Francisco last week, I thought about what I saw over the weekend. There seemed to be two Big Ideas at this conference:

1. That applications will move into the cloud/grid/utility, with everybody from Sun, Amazon, and Google to traditional hosting ISPs offering new clout/grid/utility computing solutions.

2. The big players are now moving away from software and software as a service to Platform as a Service. Yahoo, Google, Salesforce.com, Amazon, etc. are all opening up their APIs to turn them into platforms where you can come in and build your own applications on top of them.

What I Think (In Case You Cared...)

I don't think we'll ever move back to the days of dumb terminals and centralized applications run by service bureaus. Nevertheless, I was very excited by what I saw, because by making available both the hardware infrastructure and their platform APIs, the major vendors have just given us even more tools for building better applications.

Free Lessons - What I've Learned from Open Source

2008-04-11T12:22:00.000-07:00

It's been four years since I started working with open source software and started on the journey to create opentaps Open Source ERP + CRM. Sometimes I look back and think about all the things that have happened, and what I've learned from the open source experience, which is a lot, but a few things stand out:

It's All About Great Software

When you give away the code, there's nothing left to hide. People can -- and will -- judge every aspect of your software: stability, ease of use, code quality, even architecture. Because of this, open source software tends to stand much more on merits. This is why you often will see new open source projects leapfrog past projects with big-name backers or lots of hype.

There is Only One Community

It took me three years to figure this one out: when you create open source software, you're not off on an island with a band of followers. You automatically join a greater community that consists of all the other open source projects out there. The quality of your software would depend directly on the open source projects you incorporate into yours, and your success will depend on the success of the rest of the open source community.

Be Humble

Somebody recently asked me what I would do differently if I could do it all over again. It took me a few seconds before I realized the answer was "Be more humble." If you open your mind up and look at the forum posts, other open source projects, or just about anything else and say "What can I learn from it?" rather than "Why is what I have better?" you can learn a lot from the open source community.

opentaps Quarterly Update

2008-04-08T09:31:00.000-07:00

opentaps 1.0.0 Released

Last week, we officially released opentaps Open Source ERP + CRM version 1.0.0. This version marks a significant step forward for opentaps. Since the release of 0.9 nearly two years ago, opentaps has matured into a full-featured ERP and CRM suite with role-based applications designed for the sales force, customer service, warehouse, purchasing, and finance and accounting departments. We have greatly improved the usability of opentaps since the earlier versions, created a free online documentation site for it, and built a more solid foundation of automated tests that would allow us to deliver a higher quality product going forward.

A Unique Platform

Most importantly, opentaps has grown into a unique and powerful platform that brings together ERP, CRM, business intelligence, and mobile connectivity. This combination of capabilities surpasses that of almost all other commercial or open source ERP software and allows opentaps to play a much more significant role for your organization: Our flexible data model could be used to capture snapshots of practically any business activity or transaction. The synchronization tools in opentaps could bring data from such diverse sources as mobile phones, Blackberries, Outlook, and web-based e-mail and calendars such as Gmail into opentaps. Finally, the analytical tools that help you tie it all together and turn all that data into knowledge.

Together, this means that opentaps could do much more than automate back-office processes, keep accounting records, or take online orders. It could serve as the common data platform and analytical foundation of your entire organization. With opentaps, you could observe all types of activities -- sales calls, orders, shipments -- as they take place, understand their bottom-line impact, and then act upon that knowledge.

Just the Beginning

The 1.0.0 release of opentaps was a significant milestone in our four-year journey to create free and openly available business applications, but it is also the beginning of a new journey to create a whole new kind of application. Much has happened since when we began: Ajax, Web 2.0, Gmail, social networks, service oriented architectures. Many new and exciting open source projects have also appeared on the scene.

For us at opentaps, this means that we must change, sometimes in fundamental ways, to take advantage of these new resources. First and foremost, we want to make opentaps even easier to use. Under the hood, we will also be upgrading our framework to make it easier to deploy, more scalable and flexible, and more technically robust. Last but not least, we want to tap into the open-source business intelligence capabilities to help you make better business decisions. We have already started on this with the new opentaps analytics application, which is designed to help you sell more effectively by profiling your customers' behavior.

This is an exciting time for us, and I hope you will join us for the next leg of our adventure.

Enterprise Software Hierarchy of Needs

2008-03-21T12:41:00.000-07:00

There is an interesting theory in psychology called the hierarchy of needs, which said that once people's basic needs such as food and sleep are fulfilled, they would turn to higher needs for love, self-esteem, and ultimately self-actualization.

I am starting to see a similar hierarchy of needs for enterprise software, both from general trends in the industry and from our experiences with the opentaps Open Source ERP + CRM system: As organizations satisfy their most basic software needs, they tend to go up the ladder and turn to "higher" and more sophisticated needs. This "enterprise software hierarchy of needs" seems to follow this pattern:

1. Data: At the most basic level, every organization needs to standardize its operational data and make sure that everybody is using a common set of data. If this sounds simple, it isn't. In most organizations, duplicate sets of data are held in different siloed systems. For example, the accounting, shipping, and sales departments may be using separate sets of customer records. This could cause a lot of mistakes, like shipments or invoices that get sent to the wrong address. It also makes it impossible to answer basic questions like "How profitable is this customer to us?" Therefore, the first step in implementing any enterprise software is to create a common data platform with a centralized system, such as an ERP or CRM system.

2. Automation: Once you have a common set of operational data, it becomes possible to automate a lot of its processes across departments. For example, in the opentaps Open Source ERP + CRM system, you can enter an order for a customer, then process the order and invoice the customer without having to re-enter any data. Each part of opentaps would simply look up the customer, order, and pricing data it needs from the centralized database. As natural as this sounds, it amazingly is something which most companies cannot do. Instead, data is manually re-entered as an order moves from one stage of fulfillment to another.

Automation would greatly reduce manual effort and error, and anyone could see how it could make the organization more efficient. The danger, however, is that you start to automate all sorts of existing processes, even those that are losing money, while failing to focus on better opportunities. But the only way to identify the right business opportunities is through next step on our hierarchy of needs: analysis.

3. Analysis: Perhaps the greatest value of a common data platform is actually the ability to analyze the organization as a whole, rather than just blindly automating processes. This would allow you to answer some important questions, like: Which products should we sell? Or: Which customers are actually profitable?

There's an interesting chicken and egg problem here: you can only answer questions by having data, but you won't know what data you need until you know what questions to ask. Thus, while building a common data platform would help an organization analyzes performance on an ongoing basis, it may actually be better to start with a list of key questions and attempt to answer them first with legacy data and estimates. This way, we could get an idea of where the organization should be headed before implementing new software.

4. Intelligence: This truly is the holy grail for enterprise software: using all that data, automation, and analysis together to improve bottom-line results. It often relies on data to support analysis that identify the opportunities, automating processes to capture those opportunities, then collecting additional data along the way. It could be something small, like the way Amazon.com uses shopper behavior to figure out which products to show to new visitors. Or it could be something sweeping and comprehensive, like using detailed sales and profitability data to identify new lines of business. Although very few companies have achieved this level of sophistication, it is exactly what every company must do to become the leader in its field. More than streamlining some back-office processes, this should be a ultimate goal for implementing enterprise software.

In my future blogs, I will write about how the way enterprise software could fulfill these needs is fundamentally changing. I will also give some examples of how to analyze operational data to identify opportunities.

WikiPedia's Financial Difficulties and The Limits of Volunteerism

2008-03-11T08:57:00.000-07:00

Yesterday's Los Angeles Times had an article about WikiMedia's financial difficulties. It's interesting to see that a professional core team is required to sustain even this uber-example of community volunteerism. This professional core team, in turn, needs to be paid, thus requiring a business model to sustain it.

So are we looking at a limit to what community volunteerism could achieve? Does this mean any large-scale effort would ultimately require a commercial aspect to stay relevant, as many open source projects have come to believe as well?

Perhaps in the end, none of us could ever escape the need to balance our social obligations with economic realities. Not that that is a bad thing: the whole point of a market-based economy is to allocate scarce resources, so that activities don't grow beyond what could be economically justified. This may mean hard choices for individuals as we decide on whether to pursue our passions or get a job, or (hopefully) find a job that allows us to do what we love. For the greater community as a whole though, this system does ensure greater efficiency and a more rational long-term allocation of time and resources.

Using Gmail as Hosted E-mail

2008-02-25T16:58:00.000-08:00

My e-mail was broken this morning: Thunderbird is choking with the number of e-mails I get. So, as an experiment, I tried to set up Gmail as my e-mail client. This turned out to be surprisingly easy:
* I used the the Mail Fetcher to configure Gmail to pull my e-mails via POP3.
* In Gmail's Settings > Accounts screen, I added my e-mail address as a "send e-mail as" e-mail.

With just these two settings, I am now receiving and sending e-mails as sichen AT opensourcestrategies DOT com with Gmail through its web interface. This is much better than having a desktop e-mail client for me, though I found the Gmail user interface to be a bit strange.

How Gmail Works

All the e-mail client I've ever used, including Eudora, Outlook Express, Thunderbird, Hotmail, and Yahoo Mail, have the concept of a "folder." Your e-mails arrive in the inbox folder. After you send an e-mail, it gets stored in the sent folder. You can also create new folders, such as client X., and move e-mails into it to help organize your inbox.

Gmail works differently: e-mails come into an inbox, and then you can create labels for them. An e-mail could have several labels associated with it, allowing you to classify in multiple ways (client X., technical, support, etc.) After you are done reading an e-mail, you need to click on the "archive" button, so that it gets moved out of your inbox. Then you can still see it either by clicking on the "all mail" link, or by clicking on one of the labels to see the e-mails which have been tagged by this label.

In the end, I think this is better because I would be able to classify my e-mails across multiple criteria, rather than just one folder. However, it was a bit confusing when I first started using it, because the concept of an e-mail folder was so deeply ingrained in the mind of this user. I think it would be better for most users to introduce a folder concept, or at least to make the labels concept look and work more like a traditional e-mail folder.

Another interesting thing about Gmail is that it automatically groups my e-mails into threads, including both the incoming and the outgoing e-mails.

The End of Software?

A couple months ago I had considered implementing an open source Exchange solution. Now, after about 15 minutes of tinkering with Gmail, I've made that unnecessary, at least for myself. It does make me wonder, yet again, whether Gmail and Google calendar have just rendered a whole class of software, commercial or open source, irrelevant. You can bet I'll be thinking about that every time I get an email.

The Greater Community

2008-02-11T12:32:00.000-08:00

For the first couple of years after we created the opentaps Open Source ERP + CRM, we thought about our community as many other open source projects do. We defined our "community" as the sum of all the users, developers, and contributors of opentaps, and we asked ourselves questions like "What can we develop for our community?" and "How do we increase the size of our community?"

Only much later did I realize that this is completely wrong. It is wrong because an open source project is not an island, but rather part of a greater open source community that really comprises of all the open source projects out there. In that sense, there is no such thing as "our" community or "your" community. Instead we are all part of one open source community that is linked together and ultimately succeeds or fails together.

What has created this greater open source community is simply the way that open source projects utilize each other's work to create new projects. For example, opentaps directly incorporates several major open source projects, including Apache Tomcat, Derby, Geronimo, Ofbiz, as well as non-Apache projects such as Funambol, JasperReports, Pentaho, and FCKEditor. Furthermore, most users of opentaps use it with an open source database such as MySQL or PostgreSQL, and most deployments of opentaps are on Linux. Therefore, opentaps ultimately relies on all of those projects and their communities. Similarly, when you build an application with opentaps, you are not relying just on our community, but on all these other communities as well.

Once I realized this, I began to think of opentaps in a different way. Instead of thinking about what we can build for our community, I began to think more and more about what other communities could help us make opentaps a better product. Ultimately, our goal with opentaps is not to do everything under the sun, but rather to find innovative solutions for business problems. As such, the central focus of opentaps has shifted over the past year towards creating a platform that brings together the best open source projects for addressing business problems. You have already seen us incorporate new open source projects in the last six months to support business intelligence and reporting and mobility integration. This is just the beginning.

Equally importantly, I started to think about opentaps not just as a product for users of opentaps, but rather as a product that could benefit the greater open source community as well. The various opentaps applications are ultimately not just a set of screens and tables and reports, but rather a set of best practices for handling business needs. Those best practices should be available to the greater community, and one of our key goals for this year is indeed to make opentaps useful for a much larger audience of users by creating extensions or bridges to other open source projects. In other words, when you choose to use something other than opentaps, you are not an enemy. We still want to help you, and we hope we will soon.

opentaps Quarterly Update

2008-01-30T14:41:00.001-08:00

In the past few months, we have completed some important core development initiatives, while at the same time adding additional features to opentaps Open Source ERP + CRM. The significant developments include the following:

Building out a set of cross-application unit tests to verify the correctness of key areas such as order management, inventory, purchasing, and accounting.
Improved e-mail processing and management features. Customer service e-mails can now be associated with orders and assigned to different reps for processing.
Improved the use of form letters for generating outgoing e-mails and communications.
Support for managing resellers and partners, including contact information, agreements and contracts, and invoicing partners based on their sales activity.
Completed most of the documentation site for opentaps screens.
Completed the Amazon.com integration. This feature is now in production and allows the uploading of products and inventory information to Amazon, the downloading of orders, and the acknowledgment of shipments.
Completed contact, calendar, and task integration using the Funambol data synchronization server. This will enable our users to integrate opentaps with Microsoft Outlook, Mozilla Thunderbird, mobile phones, as well as Google calendar and Gmail.
Improved support for UPC codes and other product ID schemes such as ISBN throughout the application for use with barcode scanners.

Perhaps the most significant development of the past few months has been the new automated testing component in opentaps. Some of these tests are quite elaborate, involving processes in several areas of a hypothetical organization and tracking resulting changes to customers, inventory, and financial data. While the tests do not in themselves add any new features, they help us deliver on one of our original goals for opentaps: To create a stable, production ready releases which can be supported and upgraded. In addition, they give us a foundation upon which we could refine and upgrade our code base over time. Finally, the exercise of creating automated tests for applications has helped us become more methodical and rigorous in our development process.

We are now fairly close to the completion of development for the 1.0 release of opentaps open source ERP + CRM, with the only things remaining being some final unit tests and user interface polishing.

Looking Ahead

By bringing together ERP and CRM with mobility integration and business intelligence, opentaps is now a unique and powerful platform for solving business problems. The next step for us is to make opentaps more useful to a greater number of businesses and their users, so that they could unlock the potential of opentaps.

We believe the key for doing this is to make opentaps fundamentally easier: easier to use, easier to deploy, and easier to customize and extend. Just as importantly, we need to make it easier for users to analyze the data collected by opentaps to make better business decisions.

To achieve these goals, we will be making some fundamental enhancements to the technical foundation of opentaps, such as integrating opentaps with open-source content management systems to make deploying powerful, yet attractive e-commerce applications with opentaps easier, and making better use of business intelligence tools throughout the opentaps applications.

Thank you again for being part of our community, and we hope that opentaps will continue to help your business become more successful.

opentaps Framework Enhancements

2007-12-18T17:43:00.000-08:00

As we head into the new year and to the next phase of development of the opentaps open source ERP and CRM system, there are several enhancements to the framework behind opentaps which we will be making. As you can see from this list, some of these enhancements will help make opentaps easier to use and to deploy, while others should open up new areas of possibility for our users:

New Forms and Document Management Tools

We plan on replacing xsl:fo with Jasper reports and iReport during the next year so that our users could easily customize all the forms and documents that their company needs, including orders, invoices, and receipts. We also hope to introduce a way for users to manage all these documents so that they no longer have to edit XML files.

Integrating Business Intelligence

We hope to integrate the new business intelligence tools in opentaps more deeply into the applications, rather than as add-on recording tools. Our goal is to make it easy for our users to define the analysis of their data, search for information, and display reports and search results in a variety of formats, including Excel spreadsheets, PDF's, Open Office documents, and so on-- all without having to write more code.

Make opentaps a Portal

I want to find a way to make it easy to reuse and recombine the opentaps applications so that our users could do the following:

Incorporate screens and services from opentaps within other applications
Incorporate other web-based applications, including everything from Gmail to in-house applications, in opentaps.
Build custom opentaps-based applications to meet the needs of their users. For example, they should be able to build an application for an employee who does both purchasing an accounts payable by reusing the current opentaps Purchasing and Financials applications relatively easily.

Incorporate a Content Management System

We hope to be able to bundle opentaps with one or several open source content management systems to make it easier for users to build attractive front-end websites while utilizing all the capabilities of opentaps for their business management.

Rules and Workflow Engines

We hope to incorporate a rules engine and tool to support complex, pricing, commissions, and configuration-related features. We also hope to incorporate a workflow engine to allow our users to build and support complex business processes without custom code.

Through all of this, we want to make sure that we preserve compatibility for our users and for our existing applications, while the same time making opentaps both easier to use and more useful.

Beyond Open Source ERP and CRM

2007-11-28T23:16:00.000-08:00

It has been over three years since we began our road to build an open source ERP and CRM suite, and I am frankly surprised by how far we have come in such a short period of time. I owe a great deal both to our team of outstanding professional developers who work full time on developing and advancing opentaps and to the open source community at large for providing us with so many great tools.

Nevertheless, I think it is time to look further and higher--to look beyond open source ERP and CRM. I say this for several reasons:

First, "ERP" and "CRM" refer to software that was invented 15, even 20 plus years ago. The driving technologies behind classic ERP and CRM were the SQL database and client/server networks. This predates just about every other piece of software you use today. So, if we merely set our sights to build ERP or CRM software, we are really looking to build the past rather than the future of enterprise software.

Second, "ERP" and to a lesser extent "CRM" are monolithic pieces of software. They promise great things but require great effort as well--they are big, complex, and expensive to implement. This is not how software is any more. Modern software is much less intrusive, and you can get it in small incremental doses. opentaps needs to be more like that: easier to implement, so an organization can deploy it and reap the benefits of a full ERP and CRM system quickly and without as much effort.

Finally, it is a mistake to create just the "open source alternative" to existing commercial ERP and CRM systems. Reinventing the wheel is only so fun, even if it is "an open source wheel." It is far more interesting (and fun) if we could take all the innovation in all the open source communities and the best of open source software development methodologies to create simply the best software possible.

So, while classic ERP and CRM will undoubtedly be a key part of what the opentaps Open Source ERP + CRM suite provide to our users, we should not limit ourselves based on ERP and CRM systems in the market today. Rather, let's think of opentaps as a set of tools for solving business problems, and ask ourselves:

What common problems do businesses have?
What open source technologies can we use to solve them effectively?

Think outside the box: How will semantic networks help solve business problems? How will social networking impact CRM? What could peer to peer networks do? And if you have good ideas, please write!

The Limits of Open Source

2007-10-26T19:23:00.000-07:00

There was an article this week in InfoWeek entitled How Linux is Testing the Limits of Open Source Development which really hit home. In many ways, it's reflecting some of the issues that our much smaller opentaps Open Source ERP + CRM project is going through as well, and it made me realize that there may be a natural, self-correcting mechanism which limits the growth of open source projects.

Let me give you an example of how I think this works: Imagine a really smart developer sat down and started an open source project. She starts writing code, and pretty soon the software is useful. More and more people download it, and soon some other contributors/developers show up. The rate of development increases, and more features show up, leading (hopefully) to more developers. There's a positive feedback loop until the bugs also creep up. Eventually, if the bugs aren't contained, the software will become so unstable that new users would stop using it. Then all of a sudden, the project stalls out.

What to do? One logical answer is just to shrink the project down, fix the bugs, and get it stable again. This may mean the project has to be smaller, but that's no shame--open source should be about having fun writing great code, not about which project is bigger.

If, however, the project is trying to solve large-scale problems, then it must create an infrastructure similar to that found at commercial software development firms:

It would have to invest in documentation, so that both users and developers know how the application should work.
It would need to invest in a large amount of unit testing and recruit human testers, so that new features don't break existing ones.
Above all, it must create and enforce a common code culture, so that every developer could take over code written by another developer.

Such an infrastructure is necessarily less "efficient" in the sense that the lines of net code generated per developer would fall, because developer time would be taken up testing and reviewing. It also may be less fun for some developers to be reviewing code, enforcing standards, or devising unit tests rather than writing cool code. (Believe me, I tried to put it off as long as I could--but I can't any more.) Finally, it's hard for open source projects to turn away features or enforce standards. Somebody is so nice that he wants to contribute a new feature. Do you really want to hurt his feelings by saying "Your code doesn't conform to our standards?"

However, having observed several large scale open source projects up close, I'm convinced that there is no alternative if you want to build large scale open source software. We're going to have to make documentation, testing, and standards an integral part of our development process at opentaps if we want to scale up our development and deliver the features our community needs.

The Importance of Good Error Messages

2007-10-22T14:58:00.000-07:00

The following is a true story:

A few months ago, I wrote my mother-in-law a check. A couple of weeks later, it was returned to me with a letter from the bank which said: "Instrument is not negotiable."

I had absolutely no idea what that meant, so I took it, with the check to the bank. The teller had no idea, and neither did the branch manager.

Finally, a young trainee said "Oh, I know! We went over this last week in training." She then pointed out that a check is a negotiable instrument if it has five things filled in: name, numeric amount, written amount, signature, and date. In my haste to pay my mother-in-law, I forgot to fill in the date on the check.

For all you software developers who want to say "those banks are so #!@@^* !$%*^", let's try to do a better job of explaining what our software does first.

opentaps Quarterly Update

2007-10-03T15:10:00.000-07:00

As the opentaps Open Source ERP + CRM community continues to grow, I will try to publish a regular update to keep all our users, contributors, and services providers up to date with our progress.

Recent Developments

The past few months have been a period of significant changes in opentaps. By incorporating several new open source applications into our core framework, we are now positioned to transform opentaps from an ERP application to an enterprise-wide application platform. The new opentaps 1.0 will offer a full range of capabilities, including mobile connectivity and a choice of open source business intelligence tools, on top of our core ERP and CRM features.

Some of key developments include:

Integration of the Funambol Data Synchronization via the Spring framework to allow users to share contacts and calendars between opentaps and mobile phones and desktop applications such as Outlook and Evolution.
Integration of both JasperReports and Pentaho business intelligence suites into opentaps.
Integration of FCKEditor for web-based email and marketing applications.
New tools for integrating opentaps with Amazon.com, eBay, and Froogle.
Voice Over IP integration into opentaps CRM.
New opentaps Ajax UI framework tools have been developed and will now allow us to re-design much of the legacy static screens and forms.
New warehouse management applications for managing inventory, shipping (including UPS/DHL/FedEx integration), manufacturing.
New purchasing application for managing suppliers, purchase orders, and automating the procurement process.
Support for lot-level inventory management for food and beverage, pharmaceutical, and chemical industries.
New sales order entry systems in the CRM module, with enhanced support for bulk mailings and customer address validation.
Support for payroll, commissions, third party billing, and contract-based billing.
New library of tools for building online stores for opentaps with other languages and frameworks, developed in conjunction with the open source Joomla! content management project.
Last but not least, we have created a new opentaps documentation site which will be professionally developed and freely available to all opentaps users.

Coming Soon

Our immediate goals are to consolidate the significant progress of the past few months and lay the foundations for significantly greater growth in the future. As opentaps the application continues grow, it becomes imperative that we maintain consistency and quality. Meanwhile, as the opentaps community grows, we need to help new and existing users better understand how to leverage the full potential of opentaps.

Therefore, some of our most important investments right now are improving our documentation and unit testing. We are in the process of updating and re-writing many of the manuals for opentaps 0.9 on our new documentation site and plan to make it directly linkable from the opentaps, so all opentaps users could give live help from their screen. We are also in the process of implementing unit tests for some key application components.

Once we have made sufficient progress on these fronts, we will be ready to make the official opentaps 1.0 release.

Looking Ahead

Our fundamental goal remains the same: to make opentaps the enterprise application of choice for as many companies as possible.

Today opentaps is an open source ERP and CRM application with a strong out of the box feature set for product-based e-tailers, retailers, manufacturers, and distributors. We will continue to expand our offerings for these industries, but we also plan on broadening the feature set of opentaps to services companies. Some of the specific features planned are:

Fixed asset accounting
Improved manufacturing support, including enhanced Material Resources Planning and support for non-discrete manufacturing processes.
Support for services companies, including process/task and standard contract billing.
Improved marketing automation tools and supporting analytics

Simultaneously, we want to make opentaps technically more accessible, so that users could build extensions in a variety of languages and frameworks, be it Java, PHP, or Ruby On Rails. Finally, we plan to continue to improve our usability and documentation, so that it becomes increasingly easier for companies to implement and support opentaps.

Thank you again for using opentaps and being a part of our community!

Ingredients of a Healthy Open Source Projects

2007-08-10T16:03:00.001-07:00

A healthy open source project is a little like a chemical reaction and requires three key elements:

1. Users

This sounds funny, but some open source projects forget that they need users. They're written by developers based on their notions of how software "should" be, instead of for users based on what they need. The result is that funny-looking "who came up with this?" kind of a look.

This might make writing the software easier or more enjoyable for the developer, but long term it is foolish. Users are the raw material of an open source project. In one way or the other, they ultimately support the developers, whether by hiring the developers as employees or service providers or providing a pool of potential contributors and developers down the road.

2. Developers

But if there are only users, then an open source project would develop at a glacially slow pace. Projects that are short of developers are full of forum or mailing list suggestions of how things should be done, could be done, but are never done. Developers are the catalysts of an open source project. They make it happen at a pace that users could notice.

Developers need two things: something interesting to do and a way to support themselves. If the project is something that "you can't pay me enough to do", chances are it won't be a thriving open source project. At the same time, as fun as anything is to do, we all have to pay our bills, so if there isn't enough revenue for the project to support its developers, it won't thrive. This is where users need to make a conscious decision about the free software they use. While it may be to their short-term economic benefit to minimize their costs of using their software, longer term it would simply starve out the developers and leave the project moribund, rather like chopping down all the trees on their land.

An interesting note here: Just like the catalysts in a chemical reaction, some developers are good, but more developers is not necessarily better. Not any developer will do, and adding more contributors could make a project worse if it is overall not well-coordinated, and there are no good design, documentation, and testing processes in place. Developers should also be in the right portions: some coders, some documenters, a group of dedicated QA testers, and a larger group of people who help out forums or mailing lists (and actually have the right answers.) In good open source projects, they actually have formal teams for each of these.

3. Competing Products

Good open source projects are like exothermic chemical reactions: they produce their own heat or energy and keep going or growing by their own momentum. They are not endothermic reactions that require external input (ie, money) to keep going.

However, there is also some basic environmental "temperature" that's required to keep a project going, and that's dependent on the competing products in the environment. How do available commercial alternatives compare to the open source project? If they offer far richer feature set or greater ease of use, then the open source project won't attract many users, which in turn means fewer developers down the road.

Open source projects are by nature survivors. As long as there's a few (my theory is three) regular contributors, they could limp along, but it'll be like living hand-to-mouth in an Arctic circle cave. It won't be a lot of fun, and very few people would know about you. Therefore, a healthy open source project really needs enough users and developers to generate enough momentum (energy) to stay competitive with alternatives, attract more users, and eventually more developers.

My Computer in 2012

2007-07-20T09:33:00.000-07:00

As we get close to the 1.0 release of our opentaps Open Source ERP + CRM system, I've been trying very hard to think about what future versions will do.

The immediate future is pretty easy to see: we have a good system for bringing together processes and data in a company. We can easily extend it to meet the needs more of more companies and industries. We'll also be adding tools for data analysis.

But what should opentaps 4.0 do? And how should it do it?

This is hard for me to see right now, but I could envision how future computers in general should work, so I'm going to jot down a few notes to help me think later:

In 2012, my computer is the size of a credit card. I'll take it with me wherever I go and turn it on either using a scan of my eye or my thumb. It comes with no monitor and no keyboard, but once turned on, it will project a display on a surface, and I can tap and move around objects like an iPhone. I will mostly interact with it via voice, though. It should have transferred my voice print from a previous computer.

The most important tool on my computer will be my buddy list. This is actually a ranked list that is automatically generated by my computer. The most important ones are people whom I have tagged as my buddies, with the ones I frequently interact with being more important. Next down are people on my contacts list. Further down are people which I have transacted with once or twice, say on eBay. Further down from them are people whose blogs or photos I have read once.

If I search for something, my results will be filtered based on this buddy list, based on importance. Results which my buddies have liked are given the highest rankings. Results from the occasional blogger whose blog I read (like this one, perhaps?) might show up a little higher than the general muck, but not much.

My second most important tool will be a calendar. I can schedule in events myself, but it will also automatically schedule things for me. For example, if I make a dinner reservation or book a hotel, those dates will be automatically noted on my calendar. It will also automatically remind me of things. Finally, my buddies and I can share calendars, so we could see what we're each doing.

I will have an annotation tool to make notes and bookmark links wherever I go. I can annotate anything I want--a document, a web page, a scheduled event, a photograph, and they will be available.

My computer will also be a phone, camera, and video/audio recorder. I can take pictures or record conversations wherever I go. It will also interface with all the machinery of my life: my car, my refrigerator, the lights in my backyard, and so on. Before I get in my car, I can see if it needs a checkup or more fuel.

I will have all the applications I use now and more, and they will be local on my computer. My credit-card size computer will be extremely powerful, so there's no reason to use it as a dummy terminal and access basic applications over the web. I will, however, be able to share my documents, spreadsheets, and presentations with anybody I want to over the web. At the push of a button, it will synchronize my local document with a web version and make it available. It will also download changes made by others into my local version for me to consider. A sort of SVN for everybody.

Similarly, I can start jotting down notes or recording some ideas on my computer and then publish it as my blog. You'll be able to hear as well as read my blog. (Yeah!) My blog will essentially become all my records, notes, pictures, and experiences that I care to make available to others.

My photos will work the same way: I take some pictures with my computer and then can push them to the web. You will then be able to see it and then bring it into one of your albums. For example, I might take some pictures over the weekend, and you might use a couple of them for an album about Los Angeles, along with the pictures taken by other people who live here.

Except we'll no longer think of it as the web. The concept of local versus internet will be less visible. People won't say "I'm putting this on my web page." They'll say "I'm going to share this with everybody out there" or "I'll let my buddies see these pictures". The computer will deal with the web, not people. It might use an account that I subscribe to, either paid for automatically as part of my communications bill, or freely given to me in exchange for advertising rights. (Who knows? Maybe a news service would give out free accounts in exchange for publishing rights.)

I will still have a browser, but I'll use it as often as the yellow pages. The general purpose web browser will only be used to access content from providers with antiquated websites. Most of the time, I will be accessing information from service providers via what looks like the "widgets" on the Mac. For example, if I want music, I'll fire up a music widget. I can then tell it to use the music stores of my choice and select from albums, artists, and songs from it. It will manage the downloads and play the music for me, either with little headsets I carry in my pocket or beamed into my home speakers. Similarly, if I want to make travel arrangements or book a dinner, I'll fire up a little "concierge" widget. It'll subscribe to the travel services for me, look up flights, book them for me, and put them on my calendar.

My computer will be highly personalized to my needs. It'll know that I don't want to go to the same kind of restaurant twice in a week, that I don't want flights with stopovers, and that I prefer hotel rooms away from the street. I can also ask it to "hide" all my work-related tasks and appointments during the weekend.

And one more thing: I won't have to browse around through folders and look for files any more. My computer will know where to find things for me.

Another Real Life Tale of Why We Need Open Software

2007-06-08T13:46:00.000-07:00

You'll probably get bored of all my stories of why we need open software, but here's another great one:

We were working on shipping integration for the opentaps Open Source ERP + CRM system with a Global Freight Carrier. We needed to submit a ship request to them and print a shipping label. We set up an account with them, got a username, a password, an XML access key--everything is ready to go. We send the perfectly formatted XML according to their documentation requirements and kept getting an "Invalid/Missing Shipper Number" error.

We contact tech support on their website. We wait. We call our sales rep. He says there's nothing he can do. We tell him we'll ship with another equally global freight carrier. He gets his tech supervisor, who says that the only support is by email.

We wait. And wait.

And get really paranoid. We start wondering if it's because our XML is encoded in UTF-8. We start thinking about playing minesweeper with our XML, rearranging it at random until it worked.

Finally we got through! Guess what it was?

drumroll....

Our package dimensions were passed in in decimal format (ie, 3.0) instead of integers ie (3). Their system, though, kept telling us "Invalid Shipper Number" instead of "Invalid Package Dimensions."

Now, how long do you think it would have taken us to fix that if we had access to freight carrier's source code and could track the source of the problem down?

Mr. Global Freight Carrier: I'm not asking you to ship packages for us for free. Please think about how much owning proprietary shipment scheduling code is worth to you, though. And then please think about how much all of us--including you--could benefit if that code were openly available? This is just such an obvious case where everybody could benefit with open software.

Oh well, at least now I can ship with Global Freight Carrier again.

That and the fact that because the opentaps Open Source ERP + CRM is openly available to our users, and we're not inflicting this kind of pain on anybody else, make me happy.

Future of opentaps...in pictures

2007-05-19T11:10:00.001-07:00

This is a drawing for my ideas on the future of opentaps, namely:
1. A set of core opentaps ERP + CRM applications for transacting business day throughout the enterprise
2. Bringing in analytics and search tools
to help make better sense of the data
3. Integrating with devices of all stripes, for mobile phones to robots to shop floor machinery, and of course the desktop.
4. Pulling all of these components together into a unified front end.
5. Supporting it with documentation and context sensitive help.

The big unknowns are what that front end will be and how it will be developed. Would the user interface of the future be the desktop, the browser, a mobile phone, or something else entirely different? Or would there simply be more choice of how we interact with our computers in the future, and each of us would use an interface that makes the sense to us?

It's simply too early to say. Nor do we have to. Since we're an open source project, one of our basic goals is to maximize participation--make our product usable by as many people as possible. This means that we don't get to choose the browser over the desktop or framework X over Y. We have to build for all of them.

In the meantime, we know we can do a few things right now to prepare for that future. First, regardless of what the medium of interface will be, the message still has to be the same. That message for us will always be clean, consistent, and easy-to-use interfaces driven by how our users want to work. Second, we can and will have to write user interfaces that are easily abstractable in a well-tiered architecture, so that we will be able to deliver it to our users in whatever medium they choose down the road.

Future Directions for opentaps

2007-05-14T17:11:00.000-07:00

In the two years since we first created opentaps, we've had the opportunity to work with a number of companies large and small on implementing open source ERP and CRM solutions. In the process, we realized that most companies want the same things:

Bring together all the people in the organization
Automate their business processes
Record all the transactions taking place in the organization, both internal and external
Analyze that data to help make better decisions

As we had to the 1.0 release of opentaps, we've made a lot of progress in making this a reality. From Apache OFBIZ we have a very rich (and complex) data model and a scalable and stable framework for record enterprise data in large volume. We also now have a suite of opentaps modules which are designed around the way business users work, whether they are in sales, marketing, customer service, finance, warehouse, or purchasing.

We've also got a lot more to do. We need to bring in better open source tools for analyzing our data. We want to have an analytic tool which could be used to develop ad hoc reports and then incorporate those reports directly into the opentaps applications. We would like to have better search tools for data in opentaps and in other formats, such as Word or Excel. We would like to integrate with all the devices in an organization, from cell phones and Blackberries to shop floor machinery, scanners, robots, and CAD/CAM devices.

Above all, we need better user interfaces. We want opentaps to let every user in the organization do their job easily, without a lot of back and forth clicking, guessing what different screens or fields do, or forcing them to think in technical terms. Using opentaps to do your job should feel natural.

Last but not least, we want to turn opentaps into a platform that many more open source developers could leverage, whether they are working with Java, PHP, or Ruby On Rails. This means creating tools for exporting our business logic and user interface elements into any language or portal system. We hope that this would build a larger and stronger community of developers around opentaps. As part of this, our goal is also to help all these developers build successful and sustainable businesses around opentaps.

The Future of User Interfaces

2007-03-15T16:22:00.000-07:00

No, not another post about AJAX.

Instead, I'm thinking about A Digital Life by Gordon Bell in this month's Scientific American. The author is talking about the ability to set up computers and sensors to record all the experiences of a person during his or her lifetime. While I really don't want that for myself, I do think this is the future of user interface. Namely, the end of the concept of "user interface."

One day, it will seem strange that there was once a concept such as user interface. Instead, devices of all stripes will follow us around, record what we're doing, and then coordinate with each other to make our lives easier. The computer will become simultaneously ubiquitous and invisible. A terminal screen with a keyboard would be as quaint as an RCA Victorola.

If this sounds futuristic to you, then perhaps you've spent too long struggling with poorly designed user interfaces to remember how computers should work. Computers should help make our lives better. Instead, they've become the ultimate idiot-savants that require constant instructions from us: click on this screen, then that tab, locate this sub-button, and change this flag. Even the best user interfaces just make the experience more sugar-coated ("A spoon-full of sugar makes the medicine go down?") All too often, though, we have nothing better than
static screens that present data and allow us to update them. This is no more sophisticated than the software that our grandparents probably used in the 1960's.

But let's not despair: the future will be better. All we have to do is follow this basic rule: How can we help our users do what they want with the fewest clicks possible? This implies doing the following:

Build interfaces to what our users need to do, not what our programs can do.
Speak to users in natural languages.
Automate as much as possible.
Remember each user's preferences and deliver a real custom experience for them.
Integrate with outside devices.

It's All About Great Software

2007-01-09T13:18:00.000-08:00

What did we learn from 2006? For me, it's the realization that our job as developers is simply to create "great software"--software that can address the needs of real users, whether it's open source or commercial.

The open source model has several strong advantages in creating better software. It allows us to stay close to our users, listen to them, and incorporate features as they are really needed. The flexible development style and collaborative (rather than adversarial) relationship between users and developers could create better results at dramatically lower costs. In addition, our users often can help us to innovate in wholly unexpected directions.

But commercial software also has its advantages. The biggest advantage of the commercial software model is being able to spread the development costs across a large number of users. It can also support a more focused development approach and, if properly directed, bring about a good product faster. (Granted, that's a big "if.") These advantages can be especially valuable when the software is intended for non-technical users.

So it's time to get past the "open source good, commercial bad" and just focus on making better software.

A Reminder of Why Open Source Matters

2006-12-04T23:50:00.000-08:00

Sometimes you're so involved in the details, you forget the bigger picture--until something reminds you again.

Today I realized that our payment gateway processor (who shall remain nameless) does not support the Discover card correctly. When you authorize a credit card charge, the credit card (Visa, MasterCard, American Express, or Discover) performs an "address verification service" or AVS, to guard against fraud. This means matching the billing address on file of the cardholder against the address provided for the transaction. If the address and the zip code both match, AVS returns a "Y". If only the zip code matches, AVS returns a "Z". If neither matches, you get an "N", which in turn causes the gateway processor to reject the credit card.

It turns out that Discover returns an "X" if there is an exact match between the address provided and the billing address on file. Since Visa, MasterCard, and American Express do not use the "X" code, however, our gateway processor does not recognize the "X" and returns an "authorization failed" if it saw one. The result is that we cannot authorize or process Discover cards.

What to do? If you've ever programmed, you can tell this is a trivial bug. It's on the order of adding another

|| ("X".equals(avsResultStr))

to an if statement somewhere. All you have to do is dig up the code and fix it yourself--except I don't have the code!! My payment gateway processor is not open source. In fact, I don't know of a single one that is.

So I'm stuck. I can't fix it myself. There's no mailing list, no issue tracker, no community to go to either. I can raise a support request with my gateway processor. Given their past record of responsiveness, I think I'll just pass and live with buggy software.

This brings it home again why open source is better. It's not about the code that anybody else gives me. At a most basic level, it's just the right that I can fix the bugs in the software I use. If open source can do just give me that basic right, then it has made my life much better, never mind anything else.

OFBIZ Approved to Graduate from ASF

2006-11-30T15:36:00.000-08:00

Today the Apache OFBIZ project, for which I'm one of the committers and core developers, was approved to graduate from Apache incubation. It has been almost exactly a year since David Welton from the Apache Foundation first talked to me about OFBiz joining the Apache Software Foundation. At the time it almost seemed so far off and in the distance, but now that it's close at hand, I'm going to jot down a few notes about it.

What is OFBIZ?

For those of you who don't know, Apache OFBiz is a community-developed open source project for building business software: everything from high end ecommerce sites to Point Of Sales to document management to open source ERP and CRM systems, to less common things like telco billing, equipment maintenance, subscription sites. It was started in April 2001 by David E. Jones and Andy Zeneski, who met for the first time through the project (who says you shouldn't meet people online?) I got involved with the project in 2003 and became a commiter in 2005, along with several other committers from around the world. Today, we have seven committers and a few dozen contributors to the project worldwide. (Click on the link to the right or some of my other posts to learn more about OFBiz.)

How Did We Join the Apache Software Foundation?

About a year ago, David Welton, who lives and works in Italy, asked on the OFBiz mailing list whether we'd be interested in joining the Apache Foundation. We had been a successful open source project for several years now, but collectively we decided that it would be the best thing for the project to do so, even though it might be quite a bit of work. A proposal for OFBiz to join the Foundation was put to the Apache Foundation's Incubator for a vote and approved 10:1 in favor, and David Welton, Yoav Shapira from MIT, and J. Aaron Farr signed up to help mentor the project.

The process of becoming a full Apache Software Foundation project is fairly complex and involved. Since it's well described on the Apache website, I won't go into details here. Instead, let me share with you some of my own recollections of the key steps.

First, it was necessary to review all the libraries and code in OFBiz to make sure that they are compatible with the Apache 2.0 License. Since OFBiz was originally under the MIT Public License, we were already compatible with the Apache 2.0 License. However, we had to make sure that all the other open source libraries included were compatible as well. This was no small task for a project as large as OFBiz. Libraries which were released under the BSD, Apache, and MIT licenses could stay. A few LGPL libraries had to be replaced with Apache-licensed equivalents. Fortunately, David E. Jones wrote a new transaction manager using the Apache Geronimo project, and this substantial re-wiring of OFBiz was completed without a hitch in April. Jacopo Cappellato from Italy extended the form tool in OFBiz to do PDF reporting, replacing the LGPL reporting libraries. Sadly, some OpenOffice libraries which had been included for document management applications had to be moved out of the main OFBiz code base as well.

The next big thing was to get a Contribution Agreement from every contributor who had ever contributed to the project since its inception--again not a small feat, since OFBiz has always been very open about accepting community contributions, and over the years we've had over 74 contributors worldwide. When the project had been under the MIT Public License, submitting a contribution to a file with an existing MIT Public License basically means that you granted an MIT License on your contribution (check with your lawyers on this to be sure.) To join the Apache Foundation, however, all those contributions had to be vetted by having each contributor sign a Contribution Agreement which officially granted a license to the Foundation. This meant contacting every single contributor who ever contributed to OFBiz. Again, Jacopo and David took the lead on this, and over the next six months, the agreements trickled in. Everyone whom we reached was receptive. Only a couple of contributors could not be reached, and their contributions were re-coded.

Finally, new license headers and notices had to be posted to the thousands of files in the project, and new NOTICE and LICENSE files had to be created to comply with Foundation rules. Several test snapshots were created for the Foundation to vote on, and today's vote means that the members of the Apache incubator think that OFBIZ is ready to become an official Apache project. Of course, there are still more steps before OFBIZ is officially an Apache project: the approved resolution would now have to be voted upon during a full Apache Board meeting.

What Does It Mean?

For OFBiz users, it means more clearly defined licensing rights and responsibilities for the software that you would be using. More importantly, though, it means that we are committed 100% to being a community-driven open source project. In the Apache OFBiz project, everyone has an equal opportunity to contribute and shape the future of the software that they and their peers in our community would use.

For me personally, I hope that joining the Apache Software Foundation would help OFBIZ become a better open source project still. The project has grown by leaps and bounds since when I first joined, and it has gone from just David and Andy writing practically all the code to a global project with many contributors and committers worldwide. I am hoping that the Apache Foundation can help us develop the kind of structure that would allow us to harness the potential of this entire community to create great software.

Equally importantly, I hope to see greater interaction between us at OFBIZ and the developers of other Apache projects. We could certainly benefit from many of the great infrastructure projects at Apache, and our experience dealing with real world applications problems should hopefully be valuable to those developers as well. (I myself have been hoping to spend some more time on Apache James.)

No model, including the Apache model, is the perfect open source model that could solve all the software developments out there. However, I do believe that the road to better software is best traveled together. OFBIZ is living proof of that--we've built an enormously powerful enterprise applications suite from the grounds up, without a dime of Venture Capital funding. I hope that traveling with our peers at the Apache Foundation will be a lot more fun.

Where Have I Been?

2006-11-15T12:13:00.000-08:00

If you've read this blog in the past, you may have wondered where I've been all these past five months. In truth, I often wonder myself--the last five months have gone by like a blur, as we've been doing a tremendous amount of work for both the opentaps open source ERP + CRM project and for the soon-to-graduate-from-incubation Apache OFBiz project.

Happily, all is for naught. opentaps and OFBiz have both come a long way in these past months, with better features and stability and more users and contributors.

The down-and-dirty work of making an open source ERP system work for real users is not necessarily glamorous, but it's definitely given me a better perspective on the how and why of open source ERP. Some of it falls in line with what we had expected, but some of it were quite surprising--in a good way. I hope to share that with you in the coming weeks.

Ok, now back to work.

Why Enterprises Are Adopting Open Source Applications

2006-06-09T16:24:00.000-07:00

The following is based on a presentation given at the Enterprise Open Source Conference in New York City on June 6, 2006. You can download the slides for the presentation.

Good afternoon. My name is Si Chen, and thank you for coming to my presentation about the adoption of open source business applications.

It's a long of fun to be here alongside the developers and industry veterans who are shaping open source software today because, well, my background is a little different. I used to be a hedge fund manager and worked right up the street at the Met Life building on top of Grand Central, right here in New York City.

I'm here today because about six years ago, I helped start an online retail company named Gracious Style. Being a startup with very little money, we naturally went the open source route. As we grew, I wanted to help build an open source enterprise application suite that could suit the needs of our company and others like it. I did this because I fell in love with the whole idea of open source software, not just because of the cost, but because of the freedom it gave me, the user, to control my technology destiny. In that process, I met a lot of other people, from small business owners to decision makers at Fortune 500 corporations, who were all looking for the same thing—open source applications that could run their businesses.

This is our story of why we're all adopting open source business applications.

Our Open Source Projects and Why They're Significant

So first, a little background about the open source applications I'm talking about and why they are important. My talk today is based on my experiences with two closely related open source applications, Open For Business and opentaps. Open For Business is a Java business applications project that includes a framework and a set of applications ranging from ecommerce to warehouse management to manufacturing and even content management. It is developed by a global community of professional developers, of which I am one of the committers, and we are currently actively adding about 50 to 100 new features per week.

opentaps was a project we created on SourceForge a year ago to help further support the Open For Business project and our community of users. Whereas Open For Business is currently available through a Subversion code repository, we at Open Source Strategies created regular releases of the OFBiz project with upgrade scripts and professional support. We also added Financials and CRM modules which we developed to complement the core OFBiz applications so that users can get a full-featured ERP/CRM suite out of the box. Currently, opentaps is one of the top fifty projects on SourceForge.

What these applications do is give a company the ability to manage all aspects of its business, from customers to orders to inventory and manufacturing, all the way to accounting, in one application. There's no need to do integration and glue together different units of the business. Instead, it's all done with one application suite using 700+ tables that capture activity throughout the enterprise. The application is built around a very modular service-oriented architecture (SOA) which allows you to add new components or talk to other applications via web services easily. Finally, it can be deployed on either Linux or Windows and using either open source databases such as MySQL and PostgreSQL and commercial ones such as Oracle and Microsoft SQL Server.

The reason I think our project is a good barometer of what enterprises are really looking for in open source business applications is that we are the only major open source applications project that was built 100% from the ground up. We have no commercial predecessor product and no venture capitalist or investor funding. Instead, every feature you see was built by our community of mostly professional developers because somebody needed or wanted it. Thus, if you went out there all on your own and tried to start an open source applications project, you'd probably end up with something which looked a lot like our project.

Who Uses It?

So now, let's look at why enterprises are adopting applications such as ours by looking at a few organizations which are using it.

Case Study #1: Ameniti

Our first case study is a company called Ameniti Travel Clubs, which is a subsidiary of UAL Corp and a sister company of United Airlines. What Ameniti does is market travel club memberships, so for an annual fee you can get passes to the VIP lounge, the Red Carpet Club, as well as bonus miles, companion tickets, and other benefits. Ameniti uses our application to run their various travel club websites, including the Silver Wings Club, which I showed you earlier, for senior citizens. They also use it for their call centers, so when customers call to sign up, they can use it to process new applications. This can give them a fair bit of volume, since they send out 500,000 to 750,000 offers at a time in their marketing campaigns. They've done with a completely open source approach, using Linux, Apache, and PostgreSQL along with our application.

Most importantly, though, is that they're looking to our application not just for the current functionality, but as a strategic framework fot new business opportunities down the road. Indeed, when we asked them why they went with open source, these are some of the reasons they cited:

“We liked the model of cheaply acquiring adaptable technology infrastructure and customizing it for our needs.”
“Our business is very time to market driven and the framework of services provided by OFBiz greatly reduces our implementation timeframes.”
“Existing commercial solutions were older architectures, less open to customization, and encumbered from an intellectual property standpoint.”

Since what they wanted was to be able to respond quickly to new opportunities, it was very important that they be able to deliver solutions quickly and efficiently. As a reslt, they really liked the fact that they can start with a large set of features and customize it quickly and efficiently. Commercial solutions, in contrast, were not as well suited for their needs because they were harder to modify and adapt to new business needs.

Case Study #2: Snaidero

Our second case study is a company called Snaidero Engineering and Trading, and they are a subsidiary of the Snaidero Group, Italy's number one kitchen cabinet manufactuer. For those of you who have not remodeled or built a home recently, Snaidero manufactures some very high end and stylish kitchen cabinets which are also available in the United States. If James Bond had a kitchen, he'd probably have Snaidero cabinets.

Now, what Snaidero Engineering and Trading does is work with partners in countries such as Russia, Egypt, China, and Eastern Europe to help them design products and set up manufacturing facilities, and then they transfer manufacturing knowhow, or best practices, to their partners. As part of this package, Snaidero also provides their partners with manufacturing software.

What's interesting about Snaidero—and very fortunate for our project—is that they not only adopted our application but actually sponsored the manufacturing component of it. In other words, they sponsored the development of our manufacturing application and then made it open source for the rest of the community, and then they deployed the application on Windows with the MySQL database at client sites.

So why did they choose to develop an open source application rather than go with a commercial product for their partners? First, because the open source application could be customized and internationalized to suit the requirements of each market and facility. Second, because it was actually cheaper for them to develop a new manufacturing application using our framework and existing applications than to license commercial manufacturing software for each of their partners, which could really add up as they grew. Finally, going the open source route, there is no vendor lock-in to restrict what they can do down the road.

Case Study #3: British Telecom

Our thid and final example is British Telecom, one of the largest telecommunications companies in the world. British Telecom is planning to go live in the fourth quarter of this year with our application as their product catalog manager and online ordering system for mobile products and services. They will be serving all 18+ million residential and commercial customers in the United Kingdom with this system. As such, it is a very large deployment: they are scaling it out to support up to 16,000 simultaneous visitors using a cluster of 72 CPU's. Interestingly, they will be doing it with Solaris, Oracle, and the WebLogic application server, so they will be integrating our open source application with a lot of existing commercial infrastructure.

When we asked British Telecom why they are going the open source route, we got a very interesting answer. The commercial solutions they looked at were expensive, given their volume and growth rate. It would have been several millions British pounds a year. More importantly, the commercial solutions would have still required a lot of coding and development. So it's like spending a lot of money to buy one of those mail order toys, only to get a box full of little plastic parts that you have to paint, glue, and assemble. Not much fun.

Conversely, with open source, they found that it had a reasonably good fit for their requirements. It still needed work, but they thought it was a good strating point because of a “well thought out data model,” and it was “easy to change.” Best of all, it was free, so the low cost helped as well.

General Theories of Open Source Adoption

There are many other interesting companies using our applications, but I thought now we should take a look at more generally, why enterprises are adopting open source applications. So now I'd like to discuss three key drivers of open source applications adoption and then three catalysts which are speeding up the process in today's environment.

Adoption Driver #1: Open source is a better fit (the "natural selection" theory)

First, I believe that enterprises are adopting open source business applications for the very simple reason that sometimes it is the best fit. We are used to reading in the media of this titanic battle going on between open source and commercial software. Linux versus Windows. MySQL versus Oracle. But the reality is, open source and commercial software are diffferent animals. They are made differently and used differently. Therefore, it figures that there are some environments which are best suited for open source and some for commercial software.

To find out what those environments are, let's take a step back and first ask ourselves, “Why do people want open source software in the first place?” So this is a poll that we did on our website, opentaps.org, where we asked “Why are you using open source ERP software?” And the answers are not surprising: they are looking for a lower cost solution, more flexibility, and freedom from vendor lock-in, plus a host of other reasons as well.

So now let's just assume that if all of these conditions are true, then more people would choose open source software. Conversely, if none of them are true, then people would choose commercial software. In other words, if users had highly differentiated requirements, so they needed to do a lot of customization, or if commercial software is very expensive, then open source is a natural fit. If, however, the requirements are very uniform, there is little or no customization, and commercial software is very inexpensive, then more people are likely to use commercial software.

When we look at the spectrum of applications used in an enterprise, we actually see exactly this pattern. At one extreme are web applications and ecommerce or online stores. Every company has different requirements—every online store you've ever shopped at probably worked differently. As a result, you see a lot of people using open source software for these applications.

On the other extreme, you have something like accounting, which is highly standardized. There is only one way to do accounting! Anyone who tried something different has gone to jail, so there is no customization here. Also, commercial packages are relatively inexpensive. A few hundred dollars would buy you a pretty good desktop accounting package. A few thousand would get you something that would work for most organizations. As a result, we see a lot of companies still using commercial accounting software.

In the middle are applications like manufacturing and warehouse management, which are quite expensive commercially, and where different industries and companies still have fairly different requirements. Then there are applications like CRM. Again, you see a lot of differences in requirements, but there are some fairly inexpensive commercial options out there, so I put CRM between accounting and manufacturing.

When you look at our project today, you see exactly this pattern. We see a lot of people using ecommerce or developing custom web applications. Then there are some companies doing manufacturing and warehouse management, but fewer. And a smaller number still doing accounting and financials, with many more opting to integrate with existing commercial accounting packages.

So this is what I call my “natural selection theory of open source vs. commercial software.”

Adoption Drver #2: Open source empowers new business models

The second key adoption driver is that open source is a great tool for empowering new business models. If you think of a new product or service, by definition, there's probably not a commercial package supporting it yet, because, well, only you do it. Open source, by comparison, gives you a low cost foundation and flexibility for building something which would support your business models.

And what are some new business models today? Selling online, subscription-based services, personalization or customization businesses, collaborative communities. These are all examples of some new products and services for which there is often no ready commercial package. Then there is this whole “Software as a Service” or “Software on Demand” model, which I'll talk about later.

Common amongst all these business models is that they involve novel products and services, where the various vendors have highly different offerings. As a result, there is not a set of standardized “industry” requirements, because the “industry” doesn't actually exist yet. In such an environment, it is really hard to find commercial packaged software, because it's actually not economically viable to create commercial packaged software where there are no common requirements. This is where open source finds a natural niche.

Look back to our case studies, and you'll see that Ameniti, British Telecom, and Snaidero had this in common: they all had unique products or services (British Telecom), or needed to respond to new opportunities (Ameniti), or had a completely new business model for which commercial packaged software was ill-suited (Snaidero.)

Adoption Driver #3: Open source is addictive

So if this were it, and if the only reasons for open source software versus commercial software were natural selection or new business models, eventually we'd get to a bifurcated world. Some parts of an organization would use open source software, some commercial software. Newer companies would use open source, older ones commercial. But there is one more thing: open source software is very addictive. Once an organization gets comfortable with open source software, they'll want more and more of it. We've seen time and again open source get into an organization from the edge, and it's happening in applications as well.

When you think about it, this is actually very natural. Open source has the attractive advantages of being free, flexible, and giving users the freedom to do what they want. The only key hurdle is familiarity—we don't have publicists or salespeople or "targeted" mailings. People “buy into” open source software because it works!

So, as a result, we tend to see people use open source in the “edge” of their enterprise, then move it to the core over time, as it becomes proven. We've seen this with the adoption of our project—the kind of company that's most likely to adopt our open source application is one that's already using other open source software, such as Linux, MySQL, or PHP. We also see it within our application. People tend to start with ecommerce or CRM, then implement inventory or manufacturing, and finally move their accounting over to open source.

Catalysts which are Accelerating Open Source Adoption

In today's environment, there are also three catalysts speeding up the adoption of open source applications.

Transition to Service Oriented Architecture (SOA).

There's been a fundamental change in the way we built enterprise software. Instead of building it around a database, some stored procedures, and client/server user interfaces, creating a very closed and inward-looking application, we're now moving to building open architectures that can let different applications talk to each other. Thus, different commercial applications or open source ones can now all work together. As a result, SOA has created many more openings for open source applications in the enterprise.

Transition to On Demand Software.

I think this is just starting to gain momentum as a driver of open source applications and will become much more important over time, because open source applications are naturally suited for the on-demand environment. We have newer architectures with web-based user interfaces, instead of the old client/server architecture that could be very expensive to deploy. We have a flexible application that can be customized and adapted to the unique requirements of specific markets or niches, so an on-demand vendor can create a differentiated offering. Commercial software which can't be modified, in contrast, force on-demand vendors into the uncomfortable position of a commoditized product offering. Finally, open source applications such as ours can be deployed on a completely open source stack, so it's much less expensive than licensing commercial applications, databases, and operating systems.

Vendor Consolidation.

The third catalyst is a gift from the commercial software industry, and that's all the vendor consolidation that's going on today. What the consolidation is doing is reducing the amount of choice available to users. This is the essence of consolidation in any industry, be it airlines, telecommunications, or software: reduce choice, reduce capacity, and raise prices. Well, open source is the antidote of consolidation, because we're about creating more choice and lowering prices. As companies find that they no longer can meet their needs with commercial packages, they are finding open source as an alternative. Just as importantly, a new breed of independent software vendors are now rising up to serve those forgotten niches, and they are increasingly doing it with open source software as the foundation of their product offerings.

Looking Ahead: How Open Source Will Transform Enterprise Software

So in summary, we see three key drivers for the adoption of open source applications:

A “natural selection” process where open source applications are a better fit, when requirements are highly differentiated, lots of customization is required, or commercial options are very expensive.
Open source software is a natural tool for empowering new business models.
Open source is addictive for users and moving in from the edge.

We also see three catalysts for the adoption of open source software today, and they are the transition to services oriented architecture, the growing popularity of software on demand, and the consolidation amongst commercial vendors.

What's ahead? I'd like to make one prediction, and that is that open source business applications will transform enterprise software the way that Dell transformed the PC industry. If you remember way back when, people bought computers off the shelf off stores like ComputerLand, and those computers had been manufactured perhaps months earlier to vendor specifications. Then a company called Dell came along and offered customers the ability to buy a computer that exactly fit their needs. In the process, they completely revolutionized the way that computers are sold and manufactured.

I think open source applications have that same potential. We have a large pool of existing applications components and a global community of developers or assemblers, so users can either download the application and use it “as is,” the way you can buy one of Dell's most popular configurations right off the internet, or customize it to fit their unique requirements ob usiness model. Their enterprise software can be built “on demand” the way that computers can be built “on demand.” In the process, we'd have a much more efficient software industry, one that is not littered with a lot of inventory of unneeded features.

I think this will then lead to open source becoming a driver which empowers greater innovation in the real economy. All the innovation that's happening in open source software can help to power new business models and new products and services for real people. And that's what's so exciting about open source software today.

Selling Manuals

2006-05-12T11:55:00.000-07:00

Two weeks ago I spoke with someone who still doesn't believe you can make money with open source software. (Maybe the $350 million that JBoss sold for is just not enough for him?) Finally he conceded:

"Now Red Hat, they sell manuals. That business model I can understand."

Which is a very curious comment indeed, since Red Hat's manuals are right there, free to download off the Internet: see http://www.redhat.com/docs/manuals/enterprise/

In reality, documentation is a very important part of what Red Hat sells, even if you can get it for free. First and foremost, it empowers more users to take advantage of Red Hat software, and with more users, there are more contributors and more potential customers. Second, good manuals establish a bond between Red Hat the company and the end user. By educating users, Red Hat convinces them that it cares about them and can make something as complex as Linux make sense to them. And when those users want to pay for service and support, to whom would they naturally turn?

So, rather than saying "Red Hat sells manuals," it is far more accurate to say that "The manuals sell Red Hat."

(Closer to home, a few months ago I wanted to set up Mambo for the website of our opentaps open source ERP + CRM suite. I found an excellent manual by netshine.co.uk, chose them over the other Mambo service providers out there on the basis of this manual, and have been very happy with their work.)

In the spirit of things, then, today we're releasing a set of free documentation for opentaps version 0.9. These documents are culled from internal papers and documents we've created and maintained while building our applications, and we hope they will be helpful to you as you work with opentaps. While these documents, taken together, probably are over 100 pages in length, we realize that they are very incomplete and do not begin to describe all the features and potentials of our application. Nevertheless, they are a beginning.

Our long term plans for documentation is to find a collaborative platform where all the members of the opentaps and OFBiz communities can help create first-rate documentation together. We realize that this would involve a few dedicated individuals who would write the bulk of it, and commentary, additions, and proofing by the entire community. To incentivize those core documentation writers, I plan to put together a book on the opentaps ERP + CRM suite. In addition to possible royalties, such as a book should serve as a powerful advertising vehicle for the writers and their organizations . . . just like Red Hat's manuals are for Red Hat.

So that means I have to go pitch a script. Hey, I'll be like everybody else here in Los Angeles.

Merci Beacoup pour Notres Amis Francaises

2006-05-11T12:34:00.000-07:00

Life can be stranger than fiction.

On Tuesday I was on a MySQL/SugarCRM webinar, and Jacob from SugarCRM mentioned that their first translation was to French. Then, Wednesday morning, I got a translation of the opentaps CRM module into French . . . from the developers at Nereide and creators of Neogia, a sister project of opentaps developed in France.

What can I say?

Merci beacoup! Viva la France, la terre de liberte, egalite, fraternite . . . et logiciel libre!

My Sequel for Open Source ERP

2006-05-01T10:33:00.000-07:00

Pardon the pun. This is not about how to install MySQL for an open source ERP application. If that's what you're looking for, please read our Using opentaps ERP + CRM with MySQL installation guide instead.

This is about some further thoughts on the future of open source ERP and CRM applications. We were at MySQL Users' Conference last week in Santa Clara, CA and had the opportunity to speak with many different people about our open source ERP and CRM project. During those conversations and during my presentation about open source ERP, I had talked about the open source ERP proposition being the following:

Lower cost - commercial ERP software is simply too expensive
Greater flexibility - with the source code, you can really create a solution that works for you, rather than relying on ugly workarounds on top of an inflexible commercial solution
Freedom from vendor lock in - obviously no longer an issue with open source solutions.

All of the above is still true, of course, but I now think there is more to it than that. Hence, "my sequel" for open source ERP.

What I realized is that for open source to succeed, it needs to be more than just a cheaper and better alternative to commercial software. It needs to serve a need which commercial software simply does not address right now. (Tim O'Reilly pointed out a related phenomenon in his keynote at the conference as well: open source software has not replaced commercial software so much as enabled new business models.)

When you look at the most successful open source projects, this is exactly what they have done. Apache, MySQL, and PHP have become successful not by replacing commercial alternatives, but by becoming the preferred tools or platforms of choice for new web-based applications. Similarly, I now realize that open source ERP and CRM applications need to address a currently dormant need. What that need is, though, is not too clear yet. Maybe I'll think of it at some point. Or, maybe somebody in our community will come up with it first--or better, yet, implement it and send in a patch.

Whoever does would be the one who writes the sequel for open source ERP.

opentaps version 0.9

2006-04-23T07:00:00.000-07:00

This coming week, we will be releasing opentaps version 0.9 at MySQL Users Conference 2006 in Santa Clara, CA. This release is a major milestone in our quest to bring forth a powerful, flexible, well-architected, and truly open source Enterprise Resource Planning (ERP) application. It is the culmination of a three-year effort by myself, our team here at Open Source Strategies, Inc., and the entire global community of Open For Business (OFBiz) developers, contributors, and users.

What Does it Do?

Imagine if you had a very complex business. You sell custom design services to your customers, manufacture products for them, and then warranty your products. At the same time, you also sold off-the-shelf complementary products. Your products are sold online, in retail stores, and by a direct sales force to larger customers. Those larger customers require design documents and contracts.

Now imagine that your business had multiple manufacturing facilities, distribution centers, and physical stores. In different countries. With different languages, currencies, and tax laws.

Your sales force needs to track leads, convert them to accounts and contacts, track the whole sales cycle from opportunity to closing, manage their appointments, log calls and emails, and create sales forecasts. Your manufacturing plants need to schedule production, track raw materials usage, and log production times and quantities produced. Your warehouses need to track incoming and outgoing shipments, tag inventory, and prepare picking lists and shipping labels. Your warranty department needs to track requests, schedule service calls, and look up service history on each individual product. Your marketing department wants to know the conversion rates of different campaigns. Your finance departments wants status of your customers' accounts, set credit limits, and get financial results for all the subsidiaries of your organization.

opentaps is one system that could meet all of those needs.

By combining the outstanding OFBiz framework and applications, to which we are now one of the major contributors, with our own Financials and CRM modules, we have created a system that could support the ful cycle of business processes in an organization. Everything from customer acquisition (CRM and Marketing) to order taking (eCommerce, Point Of Sales, Order Entry) to the "middle" (Inventory and Warehouse, Shipping, Manufacturing, Purchasing) to Financials and Accounting.

All of this is supported in one fully-integrated application suite. There is no gorilla glue, duck tape, or Enterprise Application Integration (EAI) to hold it together. Instead, a single unified database with 730 tables captures all the activities in the organization. The application is built around a Service Oriented Architecture, and there are now over 1500 "services" or business logic components.

There is also no need to piece together your infrastructure stack. The 60+ MB download from sourceforge includes the Apache Tomcat server embedded into it, so all you have to do is unzip and start. It can be deployed with any major database, including open source databases like MySQL or PostgreSQL, on either Linux or Windows. Out of the box, it supports web and Swing user interfaces, web services, workflow, and remote-local synchronization. This combination allows you to deploy it across multiple servers over the Internet, bringing together a large and dispersed organization.

The End of the Beginning

opentaps version 0.9 is a milestone, but I see more as the "end of the beginning" rather than the completion of our quest. In particular, I'd like to see the following in future releases:

Better error message handling: Right now, some of the error messages on the screen are cryptic, like "Commit transaction failed" or "cost_comp_weff integrity violation". You'd have to scroll through the logs to see what really happened and be fairly familiar with the system to know what it means.
Support for multi-tenancy: Many people ask us if OFBiz or opentaps could be set up as a mall (think Yahoo! stores) Right now, the answer is no. It is a big application designed to support one organization with many operations, rather than many organizations squeezed into one server. I have, however, been thinking about multi-tenancy this weekend, and I think it shouldn't be too hard to extend the core framework to support it.
CRM: Full project management, time tracking, and invoicing. Right now support for this is in the framework, but I'd like to make the user interface more intuitive.
Financials: More reports. Right now we have basic financial statements, AR/AP, accounts, and reconciliations. I'd like to have more reports that allow us to run our businesses better, which leads me to the next item on my wish list.
Business intelligence: I'd like to integrate an open source business intelligence tool so business analysts can easily create custom reports to get useful knowledge about their operations. Right now, the system logs a tremendous amount of data (down to the exact history of each visit to the online store.) If we could tap into all of this data to provide real-time knowledge about what's going in a business, we could deliver a lot more value to our users.

Ultimately, though, it's the users of an open source project that drive its future, so rather than tell you what I want, it's my turn to listen and find out what you want. Thus, our main mission, even above those listed above, is to build a larger and better collaborative community around opentaps and OFBiz.

Come See Us at MySQL UC 2006

We'll be at MySQL Users Conference 2006 this coming week. We got a great spot in Community Pavilion C3, right in between Eclipse and phpMyAdmin. I'll also be giving a talk Tuesday, April 25, 2006 at 2 PM in Ballroom A. So please come see us!

opentaps at MySQL UC 2006

2006-04-10T14:15:00.000-07:00

oreilly.com got the first scoop, but we'll be officially unveiling opentaps version 0.9 at the MySQL Users Conference 2006. This will be a landmark release for us, the first one combining the Open For Business applications and framework with our Financials and CRM modules to create a complete open source ERP+CRM suite. The combined suite would give e-tailers, retailers, and manufacturers the ability to manage their entire business, from customers to orders to inventory and manufacturing to accounting, all through one web-based application.

Best of all, not only is the opentaps suite open source, but it could also be deployed with open source databases such as MySQL on open source platforms such as Linux or FreeBSD. There's no dependencies on commercial databases, operating systems, or browsers. As a result, we think this would give our users maximum flexibility and the lowest total cost of ownership.

So, if you're attending the MySQL Users Conference 2006 at the end of April, please come to the opentaps table (Community Pavilion C5) or to my talk "Opening Up Enterprise Software" on April 25, 2006 at 2 PM. We'll be doing demos, handing out some CD's, and showing you how to set up a full ERP+CRM suite using the MySQL open source database. This would be a great opportunity for us to meet and hear from some users in person, a treat since we mostly get to work with you on mailing lists.

If you can't attend or can't wait, you can preview it right now by downloading an OFBiz weekly build and checking out our Financials and CRM modules out of our SVN and building it yourself. But please come see us if you can!

Is Commercial Open Source Bad?

2006-03-20T07:12:00.000-08:00

Two recent articles, Commercial open source, a misnomer? and Free Code For Sale, may be the beginnings of a backlash against the "commercial open source" model. One challenges whether commerce and open source mix at all, while the other asks whether other models of open source development may be better for users.

It's inevitable that any major change would have its detractors. First open source was out to destroy capitalism. Now is commercial open source out to destroy open source? Is this capitalism's revenge, a case of the empire striking back? Let's look at some of the common arguments.

Commercial open source is not open source.

"Open source software" means software which is available under a certain license, as defined by the Open Source Initiative. As long as it is available under an OSI-approved license, software is open source. Being open source does not imply any other obligations on the licensor or licensee, nor does it require either to become a non-profit organization. It is inconsistent, if not hypocritical, to hail Netscape or IBM for releasing software under an OSI license and then criticize a startup for funding and releasing new software under the same license.

It is bad to use open source for marketing.

Is it in general bad for corporations to sponsor any charitable activities, including the arts, education, environment, or health care, for marketing? If you think so, then you're in the minority. Most non-profits actively solicit corporate funding to help expand their activities, and most consumers react either positively or at least are neutrally to corporations who fund charities.

Commercial Open Source Projects don't build communities.

There are two flaws in this argument. First, if the community contributes greater value to the software than it costs the commercial open source vendor to support the community, then it is rational for the vendor to support the community. It is a simple matter of economics. If the vendor does not, then it is not the commercial open source model that is bad, but rather the vendor who is doing it badly.

Second, statements like this imply a level of helplessness which should not happen with open source software. If you disagree with the commercial open source vendor's approach, go fork their source code and start your own community. Can the "community" you build outstrip their "corporate" development effort and deliver a better product for the users? If not, then their approach is probably better.

It is bad to use open source for marketing.

Is it bad to make a living with open source? If not, then is it bad for a developer to sell his services? If not, then why would it be bad for a company to sell a products? Should open source developers be limited to "making a living" with open source, or is it better that they also have a chance to build a company and "get rich"?

In the end, it's a matter of how much open source code you'd like to see. If you think developers should not be able to make a living, then we'd have very little code written only by hobbyists. If you think developers should be able to make a living, but there is no room for capital, investors, and entrepreneurs, then we'd have a cottage industry of self-employed open source developers. If, however, you do think that there is a place for commercial open source, then maybe open source could challenge commercial software as a dominant paradigm for creating and distributing software.

The Open Source Software Industry Emerges

2006-03-01T16:44:00.000-08:00

At the rate we're going, we soon won't have to wonder why there are no open source billionaires. Last year's $25 million buyout of GlueCode is dwarfed by the $222 million buyout of Snortfire. The rumored merger of Oracle and JBoss could fetch up to $400 million.

There's no question that mega-mergers of open source software companies will fundamentally change open source software, but probably not in the way you think: "Open source" itself is quite safe, but there will now be an "open source software industry," and it could be very different from what most of us in open source are used to.

The Great Idea: Merging Open Source and Commercial Software

On paper, merging open source project into a commercial software vendor is actually a pretty attractive proposition. An open source project brings a large number of users, who could help find bugs or come up with new and innovative uses for the software. Just as importantly, a free open source product essentially denies your competitors of its user base.

At the same time, a commercial vendor could bring large resources and a strong sense of direction to an open source project. It could "wrap" the open source product for enterprise users with support and marketing. Those enterprise users could in turn contribute back to the project, making it even better.

Mergers In Reality

Unfortunately, the best laid plans of mice, men, and mergers usually come to naught. The big problem is usually people, and when you merge open source and commercial software, there are lots of people. While it's a stretch to say that open source projects have no sales, marketing, legal, or product management functions, they are handled very differently at open source projects than at commercial vendors. There are also huge differences in how developers at open source and commercial vendors write code. (Believe it or not, open source code is usually better organized and easier to understand, since open source projects require highly distributed development teams.) All of these differences could lead to personality clashes that could cause a merger to come unglued.

Then there's the issue of countermoves. Most merger plans don't take into account likely competitor responses which could derail their rosy projections. Did IBM buying GlueCode cause JBoss to start talking? Possibly. If somebody bought JBoss, would BEA respond? You can bet on it.

Risks to Open Source as a Software Model

The press has naturally zoomed in on the risk of mergers to open source software. Would it mean the end of true free open source software? Would commercial vendors stop innovation and leave them dead in the water? Even more diabolical, perhaps, are the ruminations that Oracle's purchases of Sleepycat and InnoDB are meant to keep MySQL out of the enterprise database market.

In reality, most of these fears are overblown. Open source software is extremely resilient. Even if the vendor "locks up" the code after the acquisition, the existing community of users and developers can keep innovating based on the last openly available code base. If the open source developers aren't happy at their new homes, there would be plenty of users and startups who would snap them up. (A great example of this is actually PostgreSQL: In 2000, all the key PostgreSQL developers worked for a startup called Great Bridge, which subsequently went bust. These developers all found new jobs elsewhere, and the project continues apace today.)

The Risks of the Open Source Software Industry

The real "risk," if you can call it that, is that we will see boom and bust cycles in open source software, now that it has entered the big leagues. With Red Hat commanding a market capitalization of $5 billion, this is perhaps inevitable. VC investments, mergers, and an eventual IPO will just make it happen sooner.

Today's mergers, which will make open source developers, entrepreneurs, and their investors look like rock stars, could easily be tomorrow's big writeoffs, especially given the risks of merging commercial and open source firms. If this were to happen, it would cause capital for new open source startups and open source projects to dry up. Then, like all the other capital intensive industries--real estate, oil and gas, and commercial software--there will be years of boom and years of bust in open source. Some years lots of new open source projects will show up, funded by eager investors or corporate parents. Other years, we'd be counting on our old academic standbys for free code.

Are we anywhere near a bust? Probably not. The open-commercial mergers of today are still puny by corporate standards. IBM's buyout of GlueCode is like someone making $90,000 spending $25. Even if Oracle were to acquire JBoss for $400 million, it is still not taking a big risk, since it has nearly $13 billion a year in revenues.

Perhaps the open source industry is still not big enough yet--we'll need some more boom years ahead before we can really menace our commercial competitors.

Enterprise Software is Not a Refrigerator

2006-02-12T09:19:00.000-08:00

The common mistake people make when selecting ERP/CRM software, and how to avoid it.

Buying a Refrigerator

How do people buy refrigerators?

Chances are, a family sits around the kitchen table and decides what features they'd like. One wants stainless steel exterior, 30.0 cubic feet of capacity, and separate drawers for fruits and vegetables. Another wants a wine rack. Someone else wants an ice maker. An ice cream tray. A water purifier. Thus a requirements list is born.

Then you go shopping. This one has all the features but costs $1,200. That one doesn't have the water purifier or wine rack but is $300 less. Which is better? You think about it.

Is it a popular brand? What do the analysts at Consumer Reports say? How is the warranty? Do my friends use it? Do they like it?

You make a decision.

Enterprise Software is Not a Refrigerator

Usually, people buy major enterprise software, such as Enterprise Resource Planning (ERP) and Customer Relationship Management (CRM), the same way. They gather up their corporate family and create a committee of Sales, Operations, and Finance. Each draws up and prioritizes a list of features. They then go shopping: Compare vendors' offerings, prices, and support contracts. Check references. Ask other companies. Hopefully, they make a decision.

The problem, of course, is that most people know more about the refrigerator they just bought than the ERP or CRM software in which they are about to invest millions. They had a chance to open the fridge up and peek inside. Most commercial ERP/CRM vendors won't let you do that.

More problematic, though, is that this whole approach is wrong. Enterprise software is not a product you buy and use. It will become the DNA of your company. Your family will continue to cook the same food no matter which refrigerator you but (never mind what that salesperson tells you.) Your company, though, will run the way your software allows it to.

Since most companies plan on keeping their ERP/CRM software for five to ten years, sometimes longer, that software must be able to change as the companies' business does. Can an online retailer open stores successfully? Can a retail chain get online successfully? Can a retailer manufacture a private label line? Would you be ahead or behind the curve as your whole industry shifts from products to services business models, or from sales to subscriptions? You can if your software lets you.

Think these are hypotheticals? Then why did BlockBuster take so long to respond to Netflix? Why did Siebel take so long to respond to Salesforce.com?

The biggest risk of enterprise software is not a headline-grabbing failed implementation. Rather, it's an inflexible package that "succeeds" but then traps you in an outmoded business model and leaves you further and further behind as your industry or business changes.

How to Make Sure It Doesn't Happen To You

There are a few things you can do differently when shopping for enterprise software. First, you can make flexibility a priority. Ask your vendor how flexible it is. Don't take their "Yes" for an answer. Research it a little more: what is considered best practices for application architecture in the industry right now?

Indeed, there is a fundamental change going on in enterprise software right now, and that is the shift away from monolithic database-driven ERP/CRM software to a more flexible Services Oriented Architecture (SOA). SOA can (and does) fill its own books and conferences, so I'll leave details to the experts.

Better still, find out who else is using it. Instead of what you'd usually do, though, find out about companies very different from yours which are using it. If you're a manufacturer, ask about services companies using it. If you're a chain store, are there any big online retailers using it?

Then ask yourself: is this vendor flexible? Are they just responding to demand and putting in features as customers ask for them? If so, then you'd always be "in the middle of the pack." Or is your vendor actively thinking about where your industry is headed and coming up with solutions ahead of the curve to address those changes?

This is where open source may give you an advantage. Open source communities are fundamentally innovative because they can draw in ideas and contributions from a large user community. Further, you have the source code, so you can make your own modifications if your business changes. But open source is only as good as the code it produces, and that code must be built in a flexible way, following the best architecture patterns of the industry.

Remember . . .

You may never need to make toast or watch TV with your refrigerator, but you may have to run a completely different business with your ERP/CRM software in a few years.

Openness and Participation

2006-02-01T06:34:00.000-08:00

I just finished reading "Active Liberty: Interpreting our Democratic Constitution" by U.S. Supreme Court Justice Stephen Breyer. Altogether, this is a short but very interesting book and shows how a great legalist should think about important issues.

The central point of the book is that the U.S. Constitution is designed to promote the active participation of its citizens in government. As a result, the role of the U.S. Supreme Court is to interpret that Constitution with the goal of promoting greater participation. In some cases, this means protecting citizens from excessive government regulation (what he calls "modern liberty"), while in other cases, it means creating the necessary conditions for citizen involvement ("ancient liberty.")

Sometimes, the two goals require opposite courses of action, and the role of the Court is to weigh them and decide which would ultimately promote democracy. For example, campaign finance laws fundamentally restrict free speech. So are they constitutional? The answer is that it depends. The Court must very carefully consider whether a particular law, in the end, promotes greater participation by ensuring that every citizen has an adequate voice in the political process or whether they inhibit participation by preventing political dialogue altogether.

Justice Breyer also brought out an interesting historical example: in the early days after the American Revolution, there was strong sentiment against authority in the former colonies. As a result, many States enacted systems of government with very weak executive branches. Pennsylvania, for example, had a system of ten simultaneous executives (the equivalent of "presidents") who were elected only for one year each. The result, however, was not democracy but chaos. Uninformed politicians enacted laws based solely on the popular whim of the day, and there was no consistency and stability in government. The U.S. Constitution today was a response to this popular chaos and an attempt to create a system that encourages open participation by, in some cases, limiting the powers of each participant.

What does this have to do with open source? A lot.

When we create an open source community, we are essentially creating a miniature society of our own. Our goal is always to encourage participation, and key to that participation is openness. We allow others wide latitude in the use of our software and encourage their input and suggestions. But just as in a real society, there needs to be structure. We must consider the contributions of the community and set a common standard for what we accept for the whole. Otherwise, our projects, like our societies, could degenerate into chaos or fall prey to aggressive special interest groups.

When Billions Aren't Enough

2006-01-21T05:38:00.000-08:00

Our favorite anti-open source article, "Winning the Linux Wars", suggested that Microsoft partners should be "Playing the R&D card" by emphasizing that "Microsoft invests north of $6 billion a year on R&D. There is nobody in the Linux world that does that."

Well, Merck (MRK) invests about $4 billion a year in R&D. Bristol-Meyers (BMY) $669 million. Eli Lilly & Co. (LLY) $2.7 billion. Pfizer (PFE) $1.8 billion. Sanofi-Aventis (SNY) a whopping $10.2 billion, or nearly half of its $20.5 billion in revenues. Together, that's about $19.5 billion a year in research and development.

Apparently, though, that's not enough. This Friday (January 20, 2006), The Wall Street Journal's "Science Journal" ran article entitled "In Switch, Scientists Share Data to Develop useful Drug Therapies" which pointed out that there is a "crisis in 'translational science,' or turning basic discoveries into therapies," and that only twenty new drugs were approved by the US Food and Drug Administration in 2005.

One billion a drug, approximately.

More importantly, the article points out some interesting trends:

The pace of basic biomedical research is outstripping the pace of translational research. In other words, we're learning about genetics and biology faster than we're able to make drugs based on that knowledge.
As a result, foundation grants have not produced concrete results of cures for illnesses.
In response, the foundations are now shifting funds from basic research into therapies, taking over a role once left to industry.

No doubt much of the problems here is specific to the pharmaceuticals industry and the fact that we are in between sciences: the traditional pharmaceutical companies continue to research chemical cures, while basic research is focused on genetics.

Nevertheless, the fact that non-profit foundations are now moving into product development suggests that the fundamental research and product development model is encountering problems. Usually, a well-financed industry (and few are as well-financed as major pharmaceuticals: MRK, LLY, PFE, and BMY have $43 billion in cash and short-term investments between the four of them) should be ideally positioned for product development, while only basic science needs charitable support.

But there is hope: one key gene for multiple sclerosis could now be turned on and off in mice "a year sooner than they would have been, thanks to a unique collaboration that is slaying some of the most sacred cows in bio-medicine."

What is the unique collaboration? Sharing knowledge. The foundation which sponsored the research is apparently requiring the scientists it funds "share results in real time," rather than keep their discoveries proprietary. As a result, it has made the scientists feel more accountable for their work and therefore become more engaged in curing diseases.

The parallel with open source is clear. There is a real benefit to research and development when knowledge is shared. Other researchers could extend your work in new and novel ways. Furthermore, there is transparency and accountability for your work. That transparency in turn grants prestige and recognition, which are often powerful motivators. Perhaps this combination explains why academia has always been, and continues to be, a very efficient way to do research, even though their budgets are limited compared to industry.

The parallel from open source is also interesting. If open source and collaborative development is a successful model, could it help make research and development in other fields more efficient? The medical industry is sliding from open knowledge and collaborative research into a proprietary world of patents, licensing, and investors.

But if the billions aren't enough, should they be considering a return to open collaboration?

Will Microsoft Win the Linux Wars?

2006-01-09T08:01:00.000-08:00

Open source advocates should read Winning the Linux Wars in this month's Redmond Channel Partner magazine. Emblazoned with tanks and missiles, the feature article boldly proclaims that "[Microsoft] partners should relish the opportunity to compete with Linux, and they should win every time."

How? By making total cost of ownership comparisons which highlight the following advantages of Microsoft products:

More complete feature set, which could be very costly to develop in their open source equivalents.
High cost and difficulties of finding or training people with Linux skills.
A clear and coherent future for Microsoft products, thanks to its size, dominant market share, and research budget.

Some of the article's arguments are downright silly, while others point to legitimate weaknesses in the open source model as it exists today.

The Silly

For example, the article claims that "because open source software is readily customized, there's no guarantee that even a specialist fully understands a particular company's exact flavor of Linux."

Come on. This is like saying buying a popular model car is riskier because too many mechanics would know how to fix it.

The Gap

A more legitimate concern is that while open source software comes without licensing costs, it could be more expensive for a single user to complete all of its "missing" features than buying a commercial package. Despite all the advances in open source software, this actually often happens.

This may not actually be a problem. Open source software is developed on an "as needed" basis, with new features implemented as users need them. In contrast, commercial software is developed according to a product vision which could often be bloated with features most users won't ever use. Therefore, an open source solution with fewer features may actually be better for the user.

We would be deluding ourselves, however, if we thought this is always the case. There are many features which users do indeed want but do not find their way into open source software. Some of these are high performance "enterprise-class" features. Others are user-friendly convenience features. Still others are industry-specific features that fit the needs of a smaller market niche. Finally, there are the strategic "killer-app" features that a user does not know she needs today but would rush out to buy whichever software offered it tomorrow.

In these areas, the commercial software model has a fundamental advantage: commercial software vendors can spread development costs across licensing and support revenues from many customers. In contrast, a pure open source model relies on incremental contributions from a large pool of user-developers. To work at all, this model requires a large number of technically capable users, since at any one time, only a small percentage of them can be expected to contribute back to the project.

Such a development model can be highly efficient but would not address some important market needs. For example, it cannot help non-technical consumers who cannot develop the features they want themselves. (Think about 100,000 users with $10 each to spend. That's $1,000,000 of development which would probably never find its way into open source.) It also fails when, based on the laws of probability, there are simply too few users to create a large pool of developers. Finally, it would also not work very well when the users themselves could not envision the features they would actually need.

This is a serious gap which, if unsolved, could leave open source software as niche solutions for technically-savvy users who of commodity horizontal applications. If open source is to branch out and grow in other areas, new models for financing the development of open source software must be found.

We should, however, not fear: open source is the embodiment of the free market, and free markets by their nature innovate.

Our Archilles's Heel

Most importantly, "Winning the Linux Wars" correctly points out that "... the open source environment's Achilles' heel is supportability and maintenance."

There is a myth out there that there is no support for open source software, and that myth creates enormous anxiety. Lack of support means down time, down time means lost revenues and, more importantly perhaps, lost jobs. In reality, this is a myth: there often is actually better support for open source than commercial software (especially if you know how to get it.) Nevertheless, most IT managers do not like the idea of using mailing lists to track down support or expertise, so readily available support and expertise continues to be a key hurdle to open source software in the enterprise.

What It Really Means

Stepping back for a moment, though, it's not hard to see that support is really a codeword for credibility. Most buyers of commercial software don't actually verify that its features are bug free or check out its support lines. Instead, their "due diligence" consists of making sure that there are other users using the software, including, most importantly, their golf buddies.

Who then makes open source software credible? Today, it is usually the IT department of a company. Someone in IT, such as a systems administrator or developer, may try an open source application, get familiar with it and its community, and then start to convince others to use it. This person is then the ambassador or advocate for open source in the organization, and the adoption of open source software has traditionally meant winning over IT departments.

If open source is to gain popularity and move "up the stack", however, open source software will need other advocates in the enterprise. Somebody else besides the IT department must also be able to convince enterprise users that open source software is indeed a credible solution. Whether that advocate ultimately is a consulting firm, a distributor, or an ISV using open source software, we don't really know yet.

What we can be certain of is this: whoever makes open source a credible in the enterprise would ultimately win the "Linux wars."

Can Open Source Fill a Vacuum?

2006-01-04T14:40:00.000-08:00

We often hear about open source vs. commercial software, but what about when there is no commercial software?

Believe it or not, this actually happens. Last month I was at an industry tradeshow, and I walked into a conference session about software. In of itself, this was a rarity--usually, trade show attendees are too busy buying and selling to listen to software vendors. The closet you might get at one of these shows is a talk on search engine optimization or email marketing.

Even more amazing, this session was well-attended. After a while, I realized why. This industry had no commercial software vendors serving it, because it is an industry with a lot of small businesses and very complicated business processes. The former means that specialized commercial applications vendors couldn't serve them profitably. The latter means that off-the-shelf desktop accounting programs couldn't really satisfy their needs.

In this industry, people referred to their software as "Tom's program," and programmers like Tom were retiring or leaving the business. As a result, the industry is technologically a decade behind. Out of an audience of fifty, only one company was taking orders online. This is truly a software vacuum.

Can open source fill this vacuum? Possibly. Of the three groups presenting at the software session, one was a consulting firm which was using open source databases and languages to create a fairly sophisticated commercial package for this industry. Two other groups were users who had built their own in-house applications and were looking to start software development co-operatives with their peers.

This shows that there are two ways open source can fill a vacuum. First, by leveraging existing open source projects and tools, a commercial product can be created much more cheaply. Thus, commercial vendors may be able to serve these niche segments that would be un-economical for vendors who don't use open source software.

An alternative strategy is the pure collaborative or community model of software development, modeled after successful open source projects. In a sense, this is going back to the earliest days of business software, before there was commercial business software. Back then, companies would collaborate on software development by sending tapes reels in the mail. Today, collaborative development has progressed much farther with the Internet as its distribution vehicle and open source project management being much more mature. By following best practices from successful open source projects, a collaborative development effort would be much more likely to succeed.

Whether either of these models would ultimately fill the vacuum in this industry remains to be seen, but one thing is for sure: people and businesses do need software, and if there is a vacuum, open source is a logical way to fill it.

Are There Too Many Linux Distros

2005-12-06T17:17:00.001-08:00

Here's a real quote from an open source user recently:
"The more distributions the more confusing it is for the customer! You can watch that on the linux side. Microsoft is laughing while the linuxworld is diversifying instead of concentrating powers."

Question: If there were six billion Linux distributions out there, would it be good or bad for Linux?

Answer: Good. Everybody on the planet would be using and promoting his/her own Linux distro, and Linux would be the dominant operating system.

This is not as silly as it sounds. Like many markets, the dominant brand (Microsoft in this case) sells a one-size-fits-all solution for everybody, while the upstart tries to focus on specialized niches. In the case of Linux, this has happened with multiple distributions, each targeted with its own niche.

Each Linux distribution seems to have its own audience, its own personality, and its own advocacy or sales force. The Debian user may vociferously criticize Red Hat or SuSE, but enterprises which use Red Hat or SuSE probably would not find Debian agreeable. Similarly, distributions such as Red Flag Linux from China or Mandriva from France satisfy needs that US-based distributors do not or cannot.

Is this good for Linux? Ultimately, yes. Each distribution can target its own niche more effectively and form a closer bond with its users. Those users are, in turn, more likely to speak up for their favorite distributions. Just as importantly, Linux re-sellers can still sell Linux, even if they do not like one particular reseller. The net result is recruiting the largest sales force possible for Linux.

Is the competition between Linux distributions wasteful? Perhaps, but that's the cost of a free enterprise system. Is it confusing to the user? Perhaps, but comparison shopping is natural for all other products, so why should it be scary for software?

Are we all so conditioned by monopoly that even open source advocates find the signs of free enterprise threatening?

Free Software Equals Free Enterprise

2005-11-14T16:35:00.000-08:00

Despite years of support for open source, including the contribution of SapDB to MySQL, SAP may now be remembered more for Shai Agassi's comment on "IP Socialism" than anything else. Why does open source get compared to "socialism" or "communism" so much, and does it deserve it?

What's So Bad about Socialism?

Even the most die-hard conservatives don't mind a freebie from their governments. What people object to about "socialism" seems to be some combination of the following:

Forced seizure of private property through nationalization, expropriation, currency controls, or land reform.
Centralized control of productive assets and stifling of free competition.
Lack of checks and balances on the apparatchiks who control the economy.
High taxes levied on the rest of society.

Is Open Source Socialism?

Viewed in this light, open source does not share any of what most people would object to as socialism. Consider, for example, that:

Open source does not forcibly seize intellectual property. People and organizations, including SAP, license their intellectual property freely under open source licenses.
When there is no centralized control, there is usually a flowering of competition. This is exactly what has happened with open source. Think about how many Linux distributions there are. Better yet, think about how many open source desktops there are for Linux. Most people don't even realize that you can choose a desktop, but with Linux, there is GNOME, KDE, Black Ice, TWM, and countless others options.
The direct result of any free market in any economic is checks and balances. This is also true in open source: when a vendor charges too much, another one jumps. When a project is losing momentum, a fork appears to speed things up.
Finally, all of this happens without a direct tax on the user.

In fact, commercial software vendors should be careful not to take on the traits of centrally planned economies themselves. With their exclusive rights, it is far too easy for them to adopt top-down planning for future products or to use their IP ownership to exclude competition.

Far from a force for "socialism," open source could be the best defense for free enterprise in the software industry.

The Week of the Open Source Database

2005-11-10T12:00:00.000-08:00

This past week (November 7 - 11) should be officially named "the Week of the Open Source Database." In addition to the open source database conference in Frankfurt, Germany, there were these other interesting developments:

Oracle Opens Up

Oracle released a free version of their database, "Oracle Express Edition," this week. The initial license for the beta version seems to be a classic shareware license - you can test it but not use it in production, but they've promised that they'll release it under a new license that's free to develop, deploy, and distribute.

What's truly amazing about this is that according to Oracle investor relations, open source is a tiny part of the database market. Oracle has a whopping 41% market share, followed by IBM at 31% and Microsoft at 13%. All open source databases, including MySQL and PostgreSQL, are in the "Other" category which, together with minor commercial databases, command 9% of the market.

This has to be a classic case of the tail wagging the dog.

Computer Associates Punts

In other news, Computer Associates divested its Ingres database to a venture capital firm. Since open source is still such a small part of the database market (according to Oracle above), there could yet be a good place for it alongside MySQL, PostgreSQL, Firebird, and Derby. More importantly, it may also free Computer Associates up to partner with other open source databases or vendors.

But the lesson is also that it can be very hard for a commercial software vendor to reinvigorate a declining product using open source. Computer Associates reportedly spent $1 million promoting Ingres last year, apparently trying to jump start a developer community for Ingres by offering cash prizes. Maybe the new owners should consider investing in better code quality, documentation, and grass roots seminars for potential users or developers?

How Much Potential is there for Open Source Databases?

We recently did a database study of our own at Sequoia Open Source ERP: We asked people which database they would use, since Sequoia Open Source ERP is compatible with most major databases.

Over 1,760 potential users responded, and the result surprised even us. An overwhelming 90% of open source ERP users wanted an open source database. In contrast, only 7% wanted a commercial one.

This seems to point to two trends:

As people get used to open source, they'll want to use open source everywhere.
As open source business applications such as ERP and CRM mature, they'll drive even more users to open source databases.

If there is really this much potential ahead for open source databases, then it makes sense why Oracle would offer a free edition and why an investor would snap up Ingres.

Who Owns Open Source?

2005-11-07T12:19:00.000-08:00

Last week somebody actually asked me: "Does Red Hat own Linux?"

While trying to explain open source software licensing, I started to think about something even more important:

What does it mean to "own" something, anyway?

Does owning simply mean having legal title to something?

Or does it mean effective control of it?

Or, does it actually mean being able to derive economic benefit from it?

In most industries, especially capital intensive ones with high sunk costs (such as real estate, hotels, finance, and entertainment), there is a common practice of separating legal ownership from control and economic value. Investment partnerships or funds are formed to acquire assets, but control is vested in a small number of general or managing partners, and the economic benefits are divided up--usually to the advantage of those managing partners.

In fact, hedge fund managers, venture capitalists, LBO operators, movie producers, and real estate developers all share the same success secret: they are able to gain effective control and derive economic benefit from large amounts of assets without ever having to acquire legal ownership with their own money.

Viewed in this light, software is actually an anomaly: although it is a capital intensive industry with high sunk costs of development, legal ownership is not separated from control and economic benefits. In fact, in the software industry, we often have the exact opposite: legal ownership without control or economic benefit. Think about all the companies which invest huge amounts of resources in software development, copyright, and patent protection to obtain legal title to "intellectual property." Unless these companies can somehow become industry leaders, they are forever condemned to playing "catch up" to the standards someone else is setting, with very limited profit potential. A rather obvious example is all the non-Microsoft commercial office productivity suites out there.

Enter open source, which finally creates new opportunites for the software industry. Although someone may not "own" the intellectual property of open source software, he can nevertheless control its development through code contributions. He can also derive significant economic benefit from it by making it useable and beneficial to end users.

So who owns Linux? When viewed from this perspective, the answer is actually pretty complicated:

All the contributors to Linux hold the legal title to their piece of the contribution.
Linus and the active developers of Linux control the future of Linux.
Red Hat, Novell, IBM, and the other packagers, distributors, and service providers of Linux derive significant economic benefits from it by making it useful for the end user.

Open Source vs Shared or Collaborative Source

2005-10-18T15:19:00.000-07:00

With the OSI talking to Microsoft, there's been a lot of "shared source" in the news lately. The idea is pretty interesting: Paying customers (or, in some cases, significant contributors) are allowed access to the source code. Thus, the benefits of open source can be achieved without the "tragedy of the commons," where users can take from open source without ever giving back.

Is this a better model than open source? It depends:

You are able to collect payments from your users, but then you're back to selling those users again. For an established vendor, that might not be a big deal. For a startup, this cost could be significant.
If the users can make significant contributions in return, then enforcing payment to develop source code will just add the cost of "selling" to the overhead of creating software. Note that significant contributions could happen even if only a small percentage of users contributed, if there were enough users to start with.
If you're competing against a true free open source alternative, a "shared source" model would be a pretty tough sell, unless you already have a lot of brand recognition or trust amongst users. Why would anyone pay to get the source code when they can get it free somewhere else?
Even more dangerous is the possibility that after a while, the free alternative becomes so widespread that it becomes a standard. Your for-pay shared source cooperative does well for a while, then fades.

It seems to me that shared source is a step forward for established vendors in giving more participation to their users. It may also work in narrow niche markets where open source hasn't arrived...yet. For startups or broad market technologies, though, open source is probably still the better option, especially if they might end up competing against open source alternatives.

In other words, shared source probably works better for Microsoft than the rest of us.

Why Open Source Works As a Licensing Model

2005-10-12T17:29:00.000-07:00

A couple of weeks ago, there was an article in eWeek which asked whether the profit motive will "fragment" the open source community.

This actually gets us to a common misperception about open source: that it's result of a small, dedicated a group which simply believes that all software should be free. If this were true, "making money from open source" could be seen as a betrayal of the cause. Perhaps the rest of the believers would then throw up their arms, give up, or at least go elsewhere.

The reality, however, is quite different. The "open source community" is really a hodgepodge of incumbent Independent Software Vendors (ISVs), startup ISVs, hardware vendors, free speech advocates, academics, users, service providers, and, now, venture capitalists and lawyers. They have very little in common except for practicing a common software licensing model.

In light of this, the question should not be whether profit motive would fragment the "open source community," but rather why an "open source community" seems to exist at all? Why does such a disparate group get drawn to practicing this software licensing model at all? The reason is simple:

Open Source Works For a Lot of People For a Lot of Reasons

As a brief summary,

It works for developers and academics as a way to collaborate on research and development.
It works for incumbent ISVs and hardware vendors as a complementary product strategy.
It offers users and service providers a lower cost of ownership (usually) and greater flexibility (almost always.)
It offers a startup ISV a way to develop products faster and cheaper using pre-existing components and distribute those products more efficiently.
It offers free speech advocates a vehicle for creating a pool of collectively owned intellectual property.

and so forth. A full analysis of why different groups adopt open source would probably fill a small library (there are already quite a few good books about it.)

Java Goes Wild - Open Source Can Innovate!

2005-09-28T15:18:00.000-07:00

There's a column in CIO magazine this month called Java Goes Wild. Eric Knorr, one of the editors at CIO, points out that open source has created a wide ranging of new and innovative solutions to the traditional problems of Java enterprise applications. (Think Spring, Struts, Ant, Jython, JRuby, Rhino, etc. etc.) So much so, in fact, that vendors such as Sun and BEA can no longer dictate the future of the Java language.

Hopefully this will help settle the on again, off again debates about whether open source can innovate. (See Age of Innocence) If open source can't innovate, then where did all these new Java technologies come from?

How to Get Support from Open Source Mailing Lists

2005-09-23T10:29:00.000-07:00

It is possible to get outstanding support from open source communities if you know how to work with them. Of course, each open source community is different, so please follow its rules and etiquette.

How Open Source Community Support Works

Most open source communities are made up of volunteers. Its members are not paid to provide support. They're there to help each other and help the project. In recent years, as more corporations have started to contribute to open source, there are more contributors who are paid by their employers. In both cases, the contributors have a few things in common: They have a lot of pride in their project and want it to succeed, and they are also usually very busy.

As a result, it is very important on mailing lists to be respectful and to value others' time. This means that you must make it easy for them to help you by asking your questions the right way.

Asking Questions the Right Way

When I see a question on a list or forum, I usually scan and make a very quick decision on whether I would be able to help. If the question is unclear or there's not enough information, I'll usually pass--it's hard to track someone down with a mailing list. So, if you're asking a question, it is very important that you:

Get to the point quickly. In your subject and in the first paragraph, clearly describe what problem you are having. Don't use "Help me!" as your subject.
Furnish all the details. After describing the problem, describe what version you're running and what module or page you are having a problem with. If relevant, describe what platform or database you're using, and paste detailed error log messages. If you have screen shots, post them on issue trackers or upload them to your site and provide a link, but try not to clog up people's email with large attachments.
Try to do some research first. Some communities are more willing to help the inexperienced, but generally it's better if you've made somewhat of an effort to solve the problem first--for you and for others in the community.
Point out how your question might help others. Does it help answer common problems? Does it uncover a new bug?
Focus on solving the problem. Stay on topic; don't digress. Don't use your problem as an opportunity to complain.

Remember: If you make it hard to help you, fewer people will try.

Your Reputation

Longer term, how much support you receive from an open source community will depend on your reputation in it. This is no different than any other group, neighborhood, or community you might belong to. So, as you interact with others on mailing lists or forums, think about how they perceive you:

Do you ask intelligent questions?
Do you at least make an effort to figure things out?
Are you pleasant to work with?
Are you helping the project and other community members?

Most open source veterans agree: contributing back to a community, by helping others, writing documentation, or contributing code, will get you better support in turn.

General Etiquette

Finally, there seems to be a general etiquette or standard of behavior which applies to most open source communities:

Keep your questions on the list. That way, the answers are available to other community members as well. In addition, many people feel that their emails are for work or personal use, so please respect that.
Don't double post on multiple lists. If a community has multiple lists or forums, don't post to all of them at once. Try to figure out which one is appropriate.
Be civil. Don't post questions like "Your !#@* software sucks" or use headlines like "HELP! HELP! HELP! HELP! HELP!" to get attention.
Be patient. Remember others aren't paid to help you, and their help is a gift. If you can't afford to be patient, get professional support.

Why We Pay for Free Software

2005-09-21T09:03:00.000-07:00

Why a professional open source software company like ours pays for open source software.

When I first found out about open source, paying was the last thing on my mind. Why whould I ever pay for something that's already free?

Five years later, we're working daily with over two dozen open source technologies, including all the major ones such as Linux, Apache, PostgreSQL, MySQL, Eclipse, and so on. We're part of the core group of developers for the Open For Business project and sponsors of Sequoia Open Source ERP. We've become savvier developers ourselves and fairly experienced with open source software and open source communities.

We're also paying for open source software.

Í'd like to tell you that it's only because we appreciate the hard work and effort of all the open source developers out there. But there's a simpler reason: it's also how we can derive the most value from our open source software. We've learned from personal experiences that getting complex software, both open source and commercial, to produce results is not always easy. With some open source software, we've spent days, sometimes weeks, searching online or asking questions on mailing lists.

Finally, it dawned on me: I can't run a business like this. It was different when open source was my hobby, and the open source software I worked with were small. Today, though, when our business and our clients' businesses depend on open source software, I simply can't afford to take my time figuring it out. We need immediate results.

I also realized that it was extremely inefficient to learn things from scratch each time. Open source software has gotten very sophisticated and feature rich in the last few years. With it, the amount of knowledge required to configure and implement them has increased exponentially as well. As a result, it's simple faster and cheaper to pay for the help of someone who knows it well than to try to figure it out on our own. (Hey, isn't that why we all went to school?)

Finally, as an open source developer, we've learned this:

The value of software is not the source code,
but knowing what to do with that source code.

So am I disappointed? In a way, yes. I was hoping that open source software was not just "free" as in "free speech," but also "free" as in "free lunch." Now I realize that the lunch is often free, but I might have to pay for wine and dessert.

In the process of working with open source, though, I've also realized that the real value of open source software is not "free" but "control." Having access to the source code gives us control over what we can do with the software today and well into the future. It gives us the flexbility to customize it as much as we need and frees us from vendor lock-in risk down the road.

Most of all, since we have the source code, we have complete control in deciding what services we pay for and when we pay for them. We're never at anyone else's mercy.

So that's why I'm happily using free software and paying for it as we need to.

Freeware vs Shareware vs Open Source

2005-09-13T16:53:00.000-07:00

What are the differences between three models of "free" software, and why does it matter?

With all the excitement, many people are actually confusing open source software with two other models of "free" software--with potentially serious consequences. Here, we'll try to clear it up.

Freeware

The word "freeware" has been so overused, its meaning is no longer clear. Today it is often synonymous with "shareware," but for our purposes, I will define "freeware" as "software which can be downloaded, used, and copied without restrictions." (See this definition.)

Legally, the difference between freeware and open source is that you do not have access to the source code. Organizationally, this makes a big difference: There is no community and no development infrastructure around "freeware" as there is around open source software. Thus, while you can use freeware "as is," there is no real way to improve upon it or obtain support for it.

Thus, freeware is "free" as in those "Free Treadmill" classified advertisements.

Shareware

Shareware is a different concept. You can download and try shareware for free, but if you use it, you are supposed to pay for it. It is developed and released by someone who keeps full control of the intellectual property. The user does not have access to the source code and cannot modify it. There is also no collaboration or community around shareware.

In the end, the only difference between shareware and commercial software is that you can download and try shareware for free. Like commercial software, you are utlimately dependent on the developer of shareware for enhancements and support.

Thus, shareware is "free" as in "Free Sample" at restaurants or grocery stores.

Open Source

Open source means that the source code is available to all potential users, and they are free to use, modify, and re-distribute the source code. (For more details, see the Open Source Definition.) Legally, the "free" of open source refers exclusively to the source code, and it is possible to have support, services, documentation, and even binary versions which are not monetarily free. (Although some licenses, notably the GPL, requires that the source code always be freely available in such cases.)

In practice, open source usually means that the application is free to users as well as developers. Furthermore, most open source software have communities that support each other and collaborate on development. Therefore, unlike freeware, there are future enhancements, and, unlike shareware, users are not dependent on a single organization.

Open source advocates like to say that open source software is "free" as in "free speech," which is true. Since the user has the source code, it's also usually "free" as in "free lunch," even if sometimes you'd have to tip the waiter to get good service or pay for the wine.

In the Real World

The differences between the three models can be clearly seen in the kind of software that is available as freeware, shareware, or open source:

Freeware is usually a very small program, released by a student or enthusiast.
Shareware is usually a mid-sized utility or application, written by a professional developer or small software company. The developer or publisher does not have the resources to market it, so they release it as shareware with a "try-before-you-buy" business model.
Open source spans the gamut, but the largest "free" software out there are all open source--Linux, FreeBSD, PostgreSQL, Apache. Before the advent of VCs in the "free software industry," collaborative development around a shared code base was the only way a large free application could be built.

Does It Matter?

At first sight, these differences may seem like legal subtleties. In reality, though, misunderstandings about the true nature of open source can be a serious hurdle to the adoption and development of open source software.

For example, corporate users often confuse "open source" with "freeware." Thus, when we talk to them about "open source," they immediately think of the little utilities that they can download for free. Nice to have, of course, but without support or enhancements, they are dead ends for enterprise users.

(In addition, users confusing "open source" with "freeware" probably contributes to the concerns about the security of open source software. "Freeware" and "shareware" often come bundled with adware or spyware, which is actually not possible with "open source" software: see Is Open Source Secure?)

On the other hand, investors often confuse "open source" with "shareware." Thus, they are investing in companies which engage in the "free sample" business model. Many of these companies try to enforce some form of de facto if not de jure protection of their source code. Their investors may be able to reap the rewards of cheaper distribution, but, in the end, they are still investing in a traditional software vendor, with all the same risks and rewards as before.

Thus, for enterprise users to adopt open source software, they must understand the advantages of open source software over freeware. Only then will they understand that open source software does not share the same security and support problems as freeware.

Similarly, for investors to become really comfortable funding "open source business models," they will have to appreciate the potential of open collaboration in producing better software--and how it improves their risk/return tradeoffs.

Community, Corporations, and Charity

2005-09-08T10:31:00.000-07:00

Different ways for an open source community to provide for its needs and the strengths and weaknesses of each.

Every open source project has a society of users, and these socieites often have similar needs: new features, support, and documentation. Like any society, there are three ways to meet these needs:

Community - the members of the society, in this case users, get involved themselves. Each does what he can to contribute to the group. PostgreSQL and Debian seem to be organized this way.
Corporations - businesses are set up with investors, managers, and employees to satisfy the society's needs with the goal of making a profit. MySQL, Red Hat, and Sugar CRM are examples of this model.
Charity - someone provides for the society's needs without direct monetary gain, even though he may derive other benefits. OpenOffice from Sun and Mozilla during the early Netscape days are examples.

(The fourth option, "government," is thankfully absent from most open source communities.)

Community

Community effort offers the enticing advantages of empowerment, equality, and true free exchange of ideas. A veritable "liberty, equality, and fraternity" in the cyberworld.

Unfortunately, there are real limits to what community efforts can produce. First of all, volunteer efforts are dictated by what volunteers want to do. In many open source projects, this seems to be "write code" than "write documentation" or "help users." Second, community efforts depend on key organizers and ringleaders. Whole projects can be left floundering when they are not present. Finally, community efforts are harder as communities or projects get bigger, because each individual starts to feel that his contribution is simply too minute.

Corporation

Corporations can bring large resources to bear and achieve impressive results. They can also offer a greater level of reliability and predictability than unpaid volunteers. They can thus be very valuable in the ultimate success of an open source project.

On the downside, to maximize profit, corporations will usually have to assert some proprietary interests. Left unchecked, they could lead to the project becoming less and less free.

Charity

At first glance, charity would seem to be the best option. The community's needs are met without any strings attached.

Unfortunately, charitable resources are always limited, and participants have a limited role in the work of charities. (Hence the expression, "beggars can't be choosers.") Thus, charities can't be counted on to address most needs of a society at large or an open source community.

How Open Source Works

Typically, open source projects start out with charity--someone makes his work available. Then, as the project grows, a community grows up around it and begins to make more contributions. Finally, as it reaches a certain critical mass, corporations enter to provide services to the community as well.

With the renewed interest of venture investors, there's now been an inversion of this established pattern. A corporation is set up first with investor capital to create the initial work and build a community. It remains to be seen how well such corporations can live with the communities they inevitably create.

What's Right for Open Source

In a well-functioning open source society, all three components balance and complement each other. Community efforts keep corporations from asserting too much proprietary interests, while corporations bring their resources to provide services not possible with volunteers alone. It may well be that the community focuses on what it does best, such as developing software, while corporations pick up the task of support and documentation.

This has indeed been the Linux model--collaborative software development by a community, distribution and support by companies such as Red Hat and IBM. Commercial companies have popularized Linux far beyond the original developer circles, while open source communities such as Debian and CentOS keep their proprietary interests in check.

If the open source community becomes unbalanced, however, this "utopia" can easily degenerate into "distopia." If the community overly relies on charity of others, it will become a society of welfare recipients. Once those charitable resources are exhausted, the members will find themselves stranded and helpless. Conversely, if the corporations become the sole provider for a passive community, then the software will simply cease to be free.

Does Open Source Matter?

2005-08-30T15:16:00.000-07:00

The recent split of the Mambo developers raises this interesting question:

Does Open Source Really Matter?

In other words, does it matter if software is really developed by a collaborative community? Or is it good enough just to be able to get it for free?

Author Bernard Golden has written a very good perspective on this topic, entitled The Difference Between Friend and Faux Open Source. While being tactful enough not to name names, Bernard's article still gives a convincing argument that it does matter, because:

A collaborative open source community ensures the viability of commercial vendors built around it.
A truly open community protects users from vendor decisions.

If you have some time, I would highly recommend reading through it.

Open Source Mayonnaise

2005-08-12T11:00:00.000-07:00

Here's some incredibly valuable intellectual property:

1 cup oil
1 egg
2 tablespoons of lemon juice or vinegar

Beat and mix well together.

What's this?

A recipe for mayonnaise, one of the most popular condiments in America. Now you have the secret for it, courtesy of a Google sesarch for "mayonnaise recipe."

This intellectual property is even more open than open source: it's in the public domain. So how are companies like Best Foods and Kraft able to make it in the food industries with their versions of mayonnaise?

Simple: by offering predictability and benefiting from economies of scale.

For the end users of mayonnaise, like delis, restaurants, and consumers, Kraft and Best Foods offer the predictability of quality. No need to worry about it coming out too sour or too lumpy--it's the same taste in every bottle. Important when you make sandwiches all day long.

For the resellers of mayonnaise, like grocery stores and supermarkets, they offer predictability in supply. Imagine losing loyal customers because you don't have something as basic as mayonnaise in stock. For all but the largest resellers, they also offer the advantage of scale. When the a jar costs $3 retail, you won't save much by trying to make it yourself.

And how do the manufacturers themselves benefit? Through scale and distribution. By creating a vast distribution network of resellers which allows them to manufacture and sell mayonnaise in large quantities, they are able to translate a thin gross margin into respectable returns on investment.

So what if somebody had owned "mayonnaise"? Perhaps there'd be a mayonnaise billionaire somewhere. But more likely, it'd be much more expensive, many fewer people would eat it, and it'd just be another odd condiment on the bottom shelf.

The lessons for software:

There's more to a product than just knowing how to make it.
You can always make money by delivering the complete package to the end user.

Hotels and Homes

2005-08-01T17:43:00.000-07:00

After two trips in two weeks, I've thought of a simple way to compare open source and commercial software:

Commercial software is like living in a hotel room.

The software vendor is like a hotel operator. They provide you with a set of services, and you pay them. If things break, you call them. There's a certain predictability to it. On the downside, your ability to customize is limited. It may also not always be there for you, an experience we've all had when our favorite hotel has closed or been sold to new owners.

Open source software is like living in your home.

You have a lot of control over it. You can pretty much make it do anything you want (within legal limits, of course.) Your options are more open. With your own home and with open source software, there's a full spectrum of service options available, from doing it yourself to hiring contractors to everything in between.

When are hotels and commercial software right?

Hotels and commercial software are right when:

Your needs are fairly standard and well-addressed by existing service providers (hotel operators or software vendors.)
You are happy with the price they charge for their services.
You are reasonably comfortable that they'll be around (important for software) or don't really care if they aren't (usually the case with hotels.)

When is open source software right?

Like a home or its mobile cousin, the RV, open source software appeals to two distinct groups of people.

One group likes it for the low cost and don't mind either foregoing some features or doing work themselves. For example, traveling in an RV will allow you to save a lot of money, if you can do without housekeeping. Similarly, with open source software, you can save on licensing costs, if you're willing to support it yourself. Alternatively, you put together your own support package by shopping around from different vendors.

Another group has very special needs, and they can't find or don't want someone else to take care of those needs. Think about movie production trailers and surveillance vehicles--few hotels can be as accommodating. Similarly, open source software allows you to implement custom, unique, or differentiating features far better than most commercial alternatives. For many organizations which use open source software, this is a far more important reason than lower costs alone.

The future?

With the emergence of software-as-a-service (SaaS) or application service providers (ASP), will the future of software look more like hotels and homes? Will you one day "rent" generic software from SaaS or ASP providers and then use open source software as the basis for mission critical, differentiating software?

The 80/20 Myth

2005-07-19T10:56:00.000-07:00

There are three types of lies--lies, damn lies, and statistics.

- Mark Twain or Benjamin Disraeli

One commonly cited statistics about open source software is that for any given project, 80% of the code is written by ~ 20% of the developers.

Although this statement is factually correct, its interpretations are often not. For example, some have used it to suggest that the development of open source software is not that different from the development of commercial software. Others have used it to justify the excluding community participation in the development process.

These interpretations assume that the value of software is directly proportional to lines of code, which is simply not true. Consider:

The amount of time and effort required to get an 80% completed "alpha" release to a production release. How many of us have sat and waited for something that is "almost done"?
The amount of time and effort required to gather the requirements and features in the beginning of a project. Look around your own company or at your vendors. How many people actually write code versus write use cases, gather requirements, manage projects, and perform QA?
The biggest risk, and therefore one of the biggest costs, of software development is mis-specification. More software projects fail because of incorrect or inflexible specifications than any other reason.

When these factors are considered, it turns out that producing code is sometimes not even a majority of the cost of producing software.

This is where open source development processes can really reduce the costs and risks of software development. Open development processes are fundamentally collaborative. Developers and users work together to discuss what an application should do, how it should work, and how best to implement it. When a developer writes the code, others in the community help test it, report bugs, and even suggest fixes. These activities can be immensely valuable even if they result in relatively few lines of code.

Furthermore, over longer periods, the cost of maintaining software dwarfs the cost of producing the software. In open source communities, ongoing maintenance is spread over a large community of users and developers. Some will try novel uses or find new bugs. Others may think of new features and extend the application. Thus, the costs are not concentrated at any one particular user or developer organization.

The result: a better product with lower costs and less risk.

The Grand Experiment

2005-07-12T11:48:00.000-07:00

A few weeks ago, someone asked me what I did. I was at a loss for words (yes, it happens to me too.)

What do I do?

Why am I starting a company called Open Source Strategies??

Later, I realized why I was at a loss: Unlike most of my peers, I'm not betting on a particular technology. Instead, I'm conducting an experiment, which comes down to this:

Can we make software better with open source processes?

More specifically:

Can we develop better software?

Would collaboration allow us to get better ideas?
Is remote, asynchronous development more efficient?
Can a collaborative environment attract better developers? (In other words, are there smart people working outside your company?)
Can peer review help us write higher quality software, with more flexibility and fewer defects?

Can we build better software businesses?

Could software vendors:

Reduce the cost of developing software by using existing open source code?
Avoid wasteful R&D investments with a tighter vendor-user relationship?
Lower distribution costs achieved through an open source model?
Move to a demand-driven business model? (In other words, what we've been telling those other industries to do.)

Can we meet the needs of users better?

Would users benefit from:

access to source code?
greater flexbility in customization?
taking control of support/maintenance decisions?

Collaboration is at the heart of open source software. It changes the way software is created and brings together users and developers across traditional boundaries. What I'm trying to figure out is:

Can a knowledge intensive field like software benefit from collaboration?

I do not believe we can replace the commercial software business model with an open source one. Rather, I wonder if we can bring together the best of both worlds--the innovation, creativity, and efficiency of the open source model and the resources and organization of the commercial one.

Marketing gurus say everybody needs a mission statement. Here's mine:

Making Software Better

Can Open Source Survive?

2005-07-04T09:20:00.000-07:00

Thoughts on another commonly asked question about open source software.

Another common question people ask about open source software is:

Who will be around to support it?

Open source software is not singled out for persecution here. The availability of long term support is a key concern for enterprise users. Smaller commercial software vendors are often asked to disclose their financial information to prospective clients. Their larger cousins are, in turn, peppered with questions about potential mergers, acquisitions, or restructurings.

There is an added concern with open source software, though. Many users are concerned that companies which "give away" their product are repeating the worst follies of the dot-com era. In other words, will you be around to support me?

The answer is usually YES. As an open source project becomes mature, there will emerge many viable options for support. In fact, support prospects can be better for open source products, for one simple reason:

With commercial software, you depend on a single vendor for support. In open source, you can rely on a whole network of other users and suppport organizations.

With commercial software, the original vendor is the one with the source code. There may be various partners, but without access to the source code, what they can do is limited. The vendor will support its products because that's what its customers want, but it may stop doing so if:

it goes out of business;
it is acquired, and the new owners have other plans;
it decides, rightly or wrongly, that it can no longer afford to continue supporting this product;
it wants its customers to upgrade to a newer version and strategically decides to withdraw support for an older one.

In addition, remember that the focus of a commercial software vendor is generating new licensing revenue, not supporting existing installations. This is what their investors want, what their sales and marketing teams' mission is, and what their best engineers are reserved for.

In open source communities, by contrast, there is an active support ecology of users, developers, and service providers. Users and developers support each other through mailing lists, while service providers offer their clients commercial-quality support through traditional formats.

Thus, open source support is never dependent on any particular group being around. In fact, as long as there are other users, there is support for an open source project. The loss of a particular group in the support network could even be viewed as a business opportunity for others to jump in and fill a vacuum.

Now let's consider an interesting real life example:

In 1998, Netscape open sourced its browser under the Mozilla Foundation. At the time, Netscape had a multi-billion dollar market value and was one of the most influential companies in the industry. Most people would have bet on Netscape rather than the open source Mozilla Foundation to continue pushing its product forward.

Yet exactly the opposite has happened. In 2004, Time Warner which merged with AOL which bought Netscape announced that Netscape 9.0 would be the last release of this once revolutionary product. In the meantime, though, Mozilla lives in open source and has been reincarnated as the Firefox browseer. With all the security concerns about Internet Explorer, it is now gaining market share at IE's expense, in an unlikely sequel to the browser wars.

So there you have it: The flagship product of a multibillion dollar commercial software vendor has disappeared, while its sibling lives on and thrives in open source.

Now do you think open source can survive?

Is Open Source Secure?

2005-07-02T13:59:00.000-07:00

Some thoughts to one of the most commonly asked questions about open source software.

Here is a commonly voiced concern about open source software:

If anyone can contribute to open source software, doesn't it become a free-for-all full of loopholes?

At the simplest level, the answer is just NO. This statement involves a basic misconception. While anyone can propose a contribution to an open source project, any actual change must go through a small core group of maintainers first. Getting a change incorporated into an open source project is thus rather like getting an article published in a scientific journal.

At a deeper level, though, there are some real reasons why open source software can actually be more secure than their commercial counterparts. They have to do with how possible flaws are discovered and the motivations of parties involved.

Discovering Bugs and Security Flaws

Open source software is more heavily tested than their commercial counterparts. A commercial software vendor might employ one, maybe two, testers per programmer. Those testers often follow pre-written test scripts and address a finite range of use cases.

An open source project, however, might have tens or even hundreds of thousands of downloaders around the world. Each one of the downloaders may subject the software to a different use or operating environment. Any one of them can discover a bug or security flaw and submit those reports back to the project.

Once the Flaw is Found

What happens once the flaw is found?

If the code is openly available, a bug or security flaw can be definitively proven. A patch could be suggested and posted on the internet. At this point, there is only incentive to fix the flaw and no real incentive to hold back. Users would obviously want the problem solved as soon as possible. Any service provider would use it as an opportunity to demonstrate its value added by fixing its clients' installations. The project's core maintainers, while potentially embarrassed, have no further reason to cover up the flaw. In fact, if they do not fix the problems quickly enough, other community members can take matters into their own hands and start an open source derivative of the original project which incorporates the needed fixes.

In the commercial world, in contrast, things are not always so clear cut. The code is not available, so bugs or security flaws may be hard to prove. The vendor may have already made promises to its customers or statements to the press about how secure or bug-free its product is. Admitting to the bug or flaw may incur negative publicity or, worse, wrath of the marketing department. So, if the flaw cannot be definitively proven or is not generally known yet, maybe it would be better just to keep it under wraps and fix it in the next release?

But now I'm just being paranoid. This never happens in real life, right?

Finding Einstein

2005-06-21T10:24:00.000-07:00

In my last blog, I talked about how the scientific methodology, and by extension open source processes, can help attract superbly talented people. Hence, "the next Einstein."

Now let's get more specific. Whether you are a traditional closed source software vendor, a dual open/proprietary license vendor, or an open source project, these are the things you should do to attract talented developers:

Make it possible for as many people as possible to participate in your product's development. This means not just making it possible for them to view the source code but also providing tutorials and documentation to help them work with it.
Make it possible for a large group to work together remotely. Ths means setting up code repositories, mailing lists, and issue trackers and then using them. Encourage input and give feedback. Start potentially contentious yet always civilized discussions.
Train the group to work asynchronously. Each member of the group should be able to work on a small part of the project independently, without direct supervision or even contact with the rest of group.
Encourage contributions of all sizes from the group. Sometimes, great ideas come in small packages (like Einstein's papers on special and general relativity, for example.)
Institute rigorous peer review. Don't hold a popularity contest, but instead to encourage input from as many people as possible so your core review or committers' group can find the right answer.
Recognize your contributors.

Most of these suggestions are process changes, strategic or organizational in nature. There is one technical hurdle here as well: your code base must be very modular. Otherwise, it would be very difficult for new contributors to get involved, for a diverse group to work remotely and asynchronously, and for you to integrate all those contributions into a stable product.

Through this process, you can unleash the talent and creativity of a community far larger than your own company can bring together.

And you just might find the next Einstein.

The Next Einstein

2005-06-18T19:18:00.000-07:00

A hundred years ago (1905), an unknown Swiss patent clerk submitted a paper to the leading physics journal of his day. The clerk had been considered a lazy bum by his college professors and was not considered fit to work in a university.

But this clerk was Albert Einstein. His paper about special relativity fundamentally changed our view of space, time, and the universe.

That's innovation, isn't it?

But what if:

Einstein could not access existing research in physics, because it was kept "proprietary" in the universities that produced them;
He could not submit his paper because he was not at a prestigious university;
There is no way for scientists to work or contribute from remote locations;
The leading scientists of the day tried to defend their repuation through deliberate Fear, Uncertainty, and Doubt;
There was no accepted process in science of reviewing and validating any contributor's work.

Well, I'm sure you wouldn't be reading this blog right now. You'd probably be sitting in a thatch roof cottage somewhere, reading by candlelight. Becaue that was the level of technology before science took hold.

Fortunately for us, though, Einstein and countless other amateur and professional scientists can draw upon openly available knowledge, submit their ideas for peer review, and be accepted by a scientific community.

Open source can do the same thing. It is a derivative of science itself. It creates openly available knowledge, a global community, review by one's peers, and the possibility of being accepted for one's contributions. With this combination, the scientific methodology, with its proven record for innovation, can finally become a force in the software industry.

So it is quite tragic to hear of startups that talk about open source solely as a distribution mechanism and then develop software the old way. It is equally tragic that many projects only accept contributions from their own inner circle, or view other open source projects as "hostiles."

It is encouraging, though, to hear about commercial software vendors adopt and succeed with open source development processes of collaboration and peer review.

After all, don't you want the next Einstein on your team?

Reading Redmond

2005-06-13T10:49:00.000-07:00

What does a survey of the Microsoft developer community tell us?

I know we all get too much mail, but here's a magazine everybody should subscribe to: Redmond, "The Independent Voice of the Microsoft Developer Community."

In the current issue: a survey of developers' opinions towards Microsoft. Most Microsoft developers think that Microsoft is good for the industry (88%), but:

63% of respondents responded with a 7 or higher when asked if Microsoft was being arrogant (scale: 0 - 10, 10 being supremely arrogant);
51% said that Microsoft's licensing programs were unfair;
42% said that Linux, Firefox, and open source products were the biggest threat to Microsoft (another 20% said it was existing Microsoft products.)

Open source itself is not a threat to Microsoft, though--it's people's perception of open source alternatives to Microsoft products that is the real threat. More importantly, since Microsoft has always relied on the availability of products on its platform as a competitive advantage, the perception of its developers is especially important.

But open source can also help Microsoft: What if Microsoft started a few open source projects and got its developers involved? This would:

Get its developers actively involved in creating a product they would sell.
Shift some of the development cost to the developers community (admittedly, maybe not a concern for the world's biggest software company.)
Get some potentially very valuable contributions from outside the company.
Once and for all, end all those complaints about quality and security with a call to "send in your own patch!"
Make the developers feel their opinions matter.

The revenue stream from an open source project might be smaller, but the costs would be smaller too. Microsoft could still make out like bandits by offering support--which MCSD wouldn't want Microsoft to stand behind them?

Finally, Microsoft has thousands of tools and utilities that go into its flagship products which nobody really pays Microsoft for. (Would you switch to Mac OS or Linux if Windows didn't automatically zip/unzip your files?) Why not let somebody else take care of those? And make them feel more involved while you're at it?

The 30 Second Pitch

2005-06-03T05:30:00.000-07:00

I had the chance to make a "mock pitch" about open source enterprise applications to five CIO's. Here are their responses to it.

The Pitch

"I'd like to speak with you about how we can solve your problems with enterprise applications, which are: the high cost of acquisition, high cost of implementation and customization, and high cost of maintenance. These problems are exacerbated by the ongoing consolidation in the industry.

"Open source enterprise applications allow you to acquire the software for free, customize it as much as you need because you have full access to the source code, and, at the end of the day, control your own maintenance decisions."

The Responses

CIO #1: You got my attention with the "free" part. How does it work?

A: Well, you download it for free and try it out. If it works for you, you do a pilot. If that proves it can meet all of your needs, you go forward with an implementation. At the end of the day, you can make all your decisions about what to do with it--you can upgrade it when you want and get maintenace and support from whom you want. There's no vendor lock-in because you have the code.

CIO #1: You got my attention. These are definitely benefits we'd want to talk about.

CIO #2: We're really not interested in open source. I don't want to bring code in; I want vendors who provide complete solutions. I want them to take care of the generic layer for my business and my own people to focus on strategic IT that will deliver real value to our company. Open source may help us get to market faster, but I don't want in-house development to take away resources from our strategic initiatives. If there's a large organization supporting an open source solution, then we'd consider it just like anything else.

CIO #3: I'm just really concerned about whether it will be robust and secure enough to meet our needs, and how you as a vendor will survive. We have to prove to our Board that our vendors are financially viable and will be around for a while. If you're selling "free," how can you survive? I'd really need to understand your business model.

CIO #4: We use open source actively in our organization, and it's a major part of our strategy. But the evaluation is not so cut and dry. We'll have to consider it and do our due diligence in the same way as for any other solution. We'd have to study the total cost of ownership, maintenance, and support available. In some cases, we've found that an open source solution actually costs us more than commercial ones. We'd also want to support it in house, since we have a large amount of high quality IT resources available.

CIO #5: We love the idea, and I think it's the future. We don't have enough internal developers and resources for something like it right now, but we want to get there because we want to control our own destiny.

All the Answers

2005-06-03T05:07:00.000-07:00

A simple way to help you answer some common questions about open source software.

It seems we're coming back to the same questions over and over again about open source software. Fortunately, there's an easy way to think through them. We start with these observations:

Open source software was enabled by the Internet.
The Internet was originally created to facilitate scientistific research.
Open source, the Internet, and science share many common attributes.

What common attributes are those?

They all revolve around a core of openly available information.
They are highly distributed, scale-free networks of people and organizations.

So, when you get a question like "Can open source _________," just replace "open source" with "science" (or "the Internet") and ask the question again. Usually, the answer is pretty obvious. Let's try a few:

Is open source/science viable?
Yes. And it continues to gain ground.

Can open source/science innovate?
Yes. Just look around you.

Can open source/science exist without commercial rewards?
A qualified yes. Lots of science is done without any direct commercial reward, but if there were no paid professional scientists, then we'd have a lot less science. (Though it wouldn't stop altogether.)

Can open source/science co-exist with commercial rewards?
Yes. Lots of science is sponsored by companies or governments with commercial goals.

How do you making a living doing open source/science?
Most scientists make a living by transferring their scientific knowledge to others--teaching. Some make more money through consulting. Finally, a few make a lot of money by starting companies or creating products from their scientific research.
Organizations do the same thing--teaching, consulting, products/companies.

Can the products of open source/science compete successfully with commercial ones?
Scientific research in itself does not compete with commercial products. Many companies have been successful using the results of scientific research. Chances are, your company is one of these.

What are the motivations for contributing to open source/science?
For individuals: interest, ego, potential career advancement, to name a few.
For companies: it's more efficient to invest in openly available science for basic research and in-house R&D for the commercialization.

Is open source/science a "gift culture" or a "culture of abundance" rather than "culture of scarcity"?
Depends on what you mean by gift. Some scientists solely think of making a contribution to humanity. Most scientists have other motivations, and corporate and government sponsors certainly do. Yet the end result looks like a gift, doesn't it?

Can open source/science succeed without a central organization supporting it?
Yes. In fact, science seems to do better without a central organization. Unexpected contributions play a key role in the advancement of scientific knowledge. Also, the fact that there is no central organization means that science won't fail because of the failure of one company or one government.

How do advancements in open source/science happen?
Incrementally and somewhat randomly. Individuals write papers which are submitted to peer review and accepted into the scientific community. Each paper (usually) makes a small additional contribution to the total knowledge.

Can open source/science exist with a non-reciprocal license/terms?
It seems to have done fine, doesn't it? Even though you can take scientific research, make a commercial product, and never give back to science, science still moves forward at a brisk pace.

Now why don't you try a few of your own?

The Age of Innocence

2005-05-27T16:46:00.000-07:00

A colleague just emailed me an article on whether open source can innovate, where:

Larry McVoy says that open source can only "scrape together enough resources to reverse engineer stuff. That's easy... But if the world goes to 100% open source, innovation goes to zero."
Linus Torvalds is quoted to retort, "Open source actually builds on a base that works even without any commercial interest [which] is almost always secondary."

In time, we'll probably look back on these quotes with a wistful "Ahh, those were the days..." comment, when the world was younger and so we were we.

We will probably realize that open source sometimes can succeed without commercial interests, but often requires it to be really relevant. (Just look at the history of Linux: It was created by Linus the college student but flourished under Linus the head of an international effort funded and supported by major hardware and software vendors.)

We will also realize that open source can innovate, sometimes more effectively than commercial software. But this will happen when:

Open source software itself catches up to commercial software in features. Most open source projects today are still playing catch up to their commercial counterparts simply because they are newer.
The business models and processes behind open source software become more developed, probably through trial and error.
Financial markets better understand open source software and provide funding access for open source software development.

If it seems like open source cannot compete with commercial software, it's probably because open source is just too new of a phenomenon. But try to imagine the early days of the software industry itself, before there were venture capitalists, startups, IPOs, software development processes, synch-and-stabilize methods, and all the software and libraries that exist today.

Would you have believed, in those days, that a bunch of hackers and geeks playing with toy computers could one day create software that pervades every part of our lives? Would you have believed that these people could innovate?

Fortunately for us, and himself, at least one person believed: Bill Gates.

The Clone Wars

2005-05-26T09:35:00.000-07:00

A lot has been written about the IBM acquisition of GlueCode: that it's war on open source, that it validates open source, that it's just business as usual. One thing is for certain: we'll soon be seeing lots of GlueCode clones. There's even one group that's raised a fund specifically for Java middleware projects and companies using the Apache license.

Here's my guess at a plot for how it goes from here:

There are currently several frameworks, such as Spring, J2EE, .NET, and LAMP, for building (often web-based) applications. Underlying these frameworks are technologies such as PHP, Java, ASP .NET, Hibernate and products such as JBoss, WebSphere, WebLogic, Apache, etc.

These frameworks all try to streamline the process of developing web-database applications--namely, setting standards and making the process more efficient. This is happening because the underlying applications are maturing. (A website is no longer such a novelty.) Open source frameworks, including Java middleware, is a further step in this maturing process. It is essentially a step towards commoditizing the products that make up the frameworks. (This is a view that JBoss has espoused as well.)

If there are no new application needs, then the frameworks will battle themselves out in the usual manner. Debates about the various frameworks are already starting to sound like the "Pepsi Challenge."

But sooner or later a new technology will come along and create a whole new type of applications, and one particular framework (who knows which one?) will be particularly well suited for it. At that time, the demand for this new type of application will pull the appropriate framework far ahead of the rest.

Remember COBOL, FORTRAN, and Pascal? We used to have similar debates about those frameworks, until the Internet and web-based applications came along and pulled PHP, Perl, Java, and ASP .NET ahead.

The question is--what will that new technology be, and what will the next application look like? There are a few suspects: web services, utility computing, ubiquitous computing, and (a perennial contender) artificial intelligence. There could also be some hidden new technology, a phantom menance yet to be discovered.

I can't wait for the sequel...

The Developers' Utopia

2005-05-18T06:21:00.000-07:00

Not long ago, the Internet was supposed to usher in a new Era, where a globally connected economy would do away with the inefficiencies of traditional distribution. Consumers and producers would connect directly with each other in this Utopia. Whole layers of distribution would disappear. Inventory and business cycles would be a thing of the past.

Nowadays, open source is spoken of in similar Utopian terms, of which this recent Groklaw article is a good example. Open source will sweep away the inefficiencies (indeed, the inhumanity) of commercial software companies and their managements. Developers will connect directly with businesses and users through open source projects. Merit will finally replace marketing.

In reality, the Internet revolutionized business not by eliminating distribution but rather by creating new forms of distribution. Similarly, open source will not do away with management and marketing but rather create new strategies for such. Just because open source projects do not have a formal CEO and a VP of Marketing does not mean that they do not involve tremendous management and marketing skills and effort.

Indeed, the large number of abandoned open source projects on SourceForge.net seems to prove this point. A successful open source project requires real management and marketing skills as well as a serious commitment of time and effort. Going forward, the bar will probably be higher, as new projects will have to compete with either "professional" open source companies like JBoss and MySQL or major software vendors like IBM, Novell, and Oracle, which are all now getting involved with open source.

But open source does offer a tremendous opportunity for developers to advance their careers based on merit. The right way to do this, though, is usually to join an existing project, rather than trying to start one's own. Successful open source projects already have the recongition required to draw in potential clients and users. They can give developers a highly visible stage to show off their skills.

At the same time, these projects also possess the subtle but definite structures for organizing the development process and promoting the project. These marketing and management processes may be invisible for someone used to commercial software, but they must be there for an open source project to succeed.

Remember, if it sounds fun to be a "Lead Developer," imagine yourself also being the CEO and VP of Marketing for a project. And imagine not necessarily getting paid to do those jobs.

For more information, see the academic articles on my research site.

It's Happening

2005-05-12T13:31:00.000-07:00

It's gratifying and a bit scary to see your own predictions starting to come true. Here are a few headlines from the past week:

Oracle will start certifying open source applications.
IBM is buying GlueCode and getting involved with Apache's Geronimo project.
Microsoft's Steve Ballmer met with Red Hat's Szulik.

The general convergence of open source and traditional software vendors is already happening, and it will continue. It's really just a matter of time before open source becomes a part of the mainstream.

The Least You Can Do

2005-05-04T05:53:00.000-07:00

What do self-help books, get-rich-in-real-estate courses, and exercise videos have in common?

They all tell you that you can do great things, and all you have to do is just a little bit right now to get started. Usually, that little bit involves buying the book, course, or video.

Here's my version of the little bit that every software company can do with open source software. Today. Right now.

Find a few open source projects that are similar to your products and get on their mailing lists.

That's it. Easy, isn't it?

From those mailing lists, you can start seeing what questions people ask and what features they want. You can also learn a lot about users' experiences with a product similar to yours, in a way that would be difficult otherwise. Companies like Intuit and Microsoft spend millions to observe their customers in action, down to following them home and watching them install and use their software. Now all you have do to do is watch a mailing list.

In fact, an open source project's mailing list will give you a better view of users than any focus group or intense observation experiment. This is because whenever people are observed, they act differently. They tell you what they want you to hear, not what they really want. On a mailing list, though, people will actively describe their experiences, ask for help, and suggest new features because the mailing lists are the vehicles for doing just that.

Easy, isn't? And who knows? You might even decide to get involved.

A Boring Prediction

2005-05-02T09:16:00.000-07:00

Someday I'd like to be a historian. I picture myself donning a tweed jacket, wearing a sweater vest, and delivering a stirring lecture about the Napoleonic Wars or the Ming dynasty in front of a blackboard (and hopefully some students, too.)

Unfortunately, people think historians are boring (doubly unfortunately, my wife is one of these people.) The reason is that historians have a dull, analytical perspective on events. They (we?) boil events down to cut and dry technological and demographic trends, devoid of any passion, idealism, intrigue, or heroism. Who wants to read about advances in metallurgy or seed planting techniques?

So, while I bide my time, I'll perform a boring historian's analysis about open source software:

Today, open source software is seen as an epic struggle of freedom versus property, empowerment versus establishment, even community versus capitalism. It is described as a fundamentally disruptive force, a transition to a gift culture, etc. Even manifestos are written about open source software.

When we look back a few years from now, though, we will realize that open source software was simply the result of two bigger trends:

The internet, which has created a globally connected software industry; and
The "blog generation," a more general demographic trend of greater user and consumer participation across all major industries.

The former makes a highly distributed model of production and distribution possible. The latter has made users activley participate in the development of their software.

That's it. Nothing more, nothing less. If this sounds boring to you, well it is. In time, open source will be a normal part of the software industry, indeed all technology industries. Having an open source strategy in, say 2010, will be no more groundbreaking than having a website today.

And here's my boring prediction:

In five years time (2010), there will no longer be a meaningful distinction between "open source" and "closed source" software vendors. All major software companies will:

Actively support some open source projects;
Actively adopt and use some open source projects as part of their products; and
Fight off some other open source projects' assaults on their products.

What about Microsoft, you ask? Well, in fact, they are already doing all three.

Two Can Play this Game - Trial By Fire 2

2005-04-18T05:49:00.000-07:00

Part two of a continuing series on unique business challenges facing open source projects and businesses.

Let's say that you have a successful open source project or business: Customers love the low cost and flexbility. Venture capitalists are beating a path to your door. Even slashdotters think you're cool.

Then, one morning, you wake up and find that--

A commercial ISV waives licensing fees on a competing product.

Think it can't happen? Ask yourself this:

How many razor companies give away the razor to sell blades?
How many printer companies give away the printer to sell cartridges?
How many cell phone comopanies give away the phone to sell services?

In every industry, imitation is the sincerest form of capitalism. If there really are enough profits from services to forego licensing, how long would it take your competitors to figure it out as well?

This is a risk every open source project and open source software company needs to plan for.

If your open source strategy is just about the lowest price, then you have a real problem. Once your competitors respond by waiving licensing costs, what advantage do you have?

Fortunately, open source can be about more than just price. Because it creates a high level of user involvement, an open source project can provide some unique benefits that are priceless. For example:

Can community testing and patches help you produce higher-quality software?
Can community input focus your development efforts and make you more efficient?
Has the open source development process made your code more modular and cleaner?
Does your source code really offer customers greater flexibility?
Can you build a global, distributed development model and innovate faster, better than your competition?
Are your users more loyal because they are part of a real community, rather than just passive licensees? Will they go evangelize for you?

In other words, do you have a real community, or do you just offer free stuff? This will determine whether your project (and company) is more like Amazon.com or a Pets.com.

In the end, your commercial competitors may be doing you a favor by waiving their licensing fees. They will draw away users who were just interested in "free." Now you can focus on a core customer base which really appreciates the true value of your offering. By re-focusing on its needs, you will eventually emerge stronger.

Open Source Transitions - Trial By Fire 1

2005-04-18T05:48:00.000-07:00

Part one of a continuing series on critical challenges that are unique to open source projects and businesses.

In the beginning, an open source project is created by a single developer, a small group, or a startup. The initial creators write the code, put it out there, and hope that others will adopt it. In the meantime, they do everything from designing new features, writing code, reviewing and committing them, fixing bugs, writing documentation, and providing support on mailing lists.

Unfortunately, it usually ends here. Most open source projects languish and are eventually abandoned by their creators.

Fortunately, though, this open source project succeeds. Many users download it and start working with it. The mailing lists get busy. After a while, a community develops around the project. The original creators bask in the glory of their successful project and are (hopefully) successful themselves from it, selling everything from books to consulting services to value-added commercial products.

Then, the project comes to a transition point. As community members become more involved with the project, they are ready to play a bigger role. They start planning and developing new features themselves, writing their own documentations and how-to guides, answering emails on the mailing lists, and so on.

What happens now?

At a minimum, the project creators need to pull back from their earlier hands-on roles. They need to let others do the day-to-day work of fixing bugs, writing documentation, and providing support. At the same time, they must still shepherd the project. Only they can set the right direction for the project. They still must invest in strategic development to push the project along. They must maintain high standards for the documentation, code, and support. And they must do all this while continuing to build and nurture the community.

This is not easy to do. It is very similar to the transition that most entrepreneurs face as their businesses grow and they must relinquish day-to-day tasks to focus on the bigger picture. Many entrepreneurs can't or doesn't want to do this and eventually move on.

This transition is even harder when the original creators have built businesses around their project. They may be selling support services, documentation, or commercially-licensed versions with premium features based on their open source project. What happens when the community starts to create free (or nearly free) alternatives to the founders' offerings? Here are some options:

Squash the competition. The knee jerk reaction. Use your position as the maintainer of the project to shut out competing efforts. Or, spread FUD around community efforts, with statements like "It's so great to see all this involvement, but..." But is this productive? Should you really be expending valuable resources to create something that would otherwise be free?
Keep competing offerings. Inertia at work. Even though the community has produced a free alternative, you continue to spend money to develop, maintain, and sell your version. Hopefully, you can leverage your position in the community (essentially, your brand) to do so. You're still paying for something that should be free, though. And if your project continues to be successful, the community will eventually grow large enough to overtake you. In that case, your expensive but inferior offerings could actually end up damaging your brand. (This has happened to a major vendor whose name I will not mention here.)
Replace your offering with the community's. Like Gordon Gekko said in Wall Street, "Never get emotional about a stock." Once better code, features, documentation, or support become available from the community, drop yours and adopt its. This would, however, significantly affect your ROI. It may also embolden others to start competing with you.
Give back to your community. Periodically gift proprietary code and documentation back to the community. You can create goodwill, enhance your brand, and eliminate potential competitors. But then how do you convince someone to continue to pay for proprietary offerings when, sooner or later, you might make it free?

The challenges in managing this transition successfully are:

Can the original creators of the project let go of their own ego and emotions and let others play a bigger role?
Can they encourage the right people to get involved? Can they filter out weak ideas while avoiding a fork in the project?
Can they repeatedly re-focus their resources correctly on core versus non-core competencies? Can they successfully extract value from their core strengths, while getting others to produce non-core offerings?

This is not an easy transition for any entrepreneur. It is especially difficult with open source projects, since people don't have to work with you.

The Next Microsoft

2005-04-15T05:32:00.000-07:00

Apparently, a popular question asked of hopeful software start-ups is:

How will you become the next Microsoft?

One imagines a young, optimistic entrepreneur, slightly flustered, starting to talk about how he will eventually one day own a dominant technology platform. Ahh, the dreams of youth.

It is a very natural question to ask—Microsoft is without doubt the most successful software company of all time. It owns the core platform for PCs with Windows and Office, dominates access to the Internet with Internet Explorer, has a large presence online with MSN, and commands a vast following in the developer community with Visual Basic, C#, and Visual Studio tools. Why wouldn't someone want to be the next Microsoft?

Imagine, though, if someone had asked the young Bill Gates and Paul Allen:

How will you become the next IBM?

Did two flustered youngsters describe a strategy to build a world-dominating hardware business and bundle software and services along with it?

Strange as it may seem today, this question was just as natural back in those days. IBM had dominated all of computing, with its mainframe computers and hardware, OS/360, business applications, and services. It was so dominant that it too was the subject of lengthy anti-trust litigation. And even in the late 1980's, people spoke of "starting the next IBM."

Indeed, someone must have been asking this question back in the mid-1970's, because most successful startups of that era, including DEC, Wang, Data General, followed the IBM model and offered bundled hardware, operating systems, applications, and services. Microsoft was actually an oddity: not only did it specialize in the then non-existent personal computer sector, it only produced software. The rest, as they say, is history.

In fact, both IBM and Microsoft are products of their times. The IBM model was a product of the 1950's, when computing itself was in its infancy. It bet big on the very concept that businesses would want computers and produced such great hits as the System/360 mainframes, OS/360, JCL, MVS, COBOL, etc.

The Microsoft model was also a product of its time, this time the late 1970's. Microsoft realized that, unlike IBM's big corporate computing customers, the PC user will care a lot more about ease of use and features. It bet big and won big with DOS and then Windows, standardized the nascent personal computer industry, and used that standard to its fullest advantage.

Similarly, today's startup must be a product of our time. So, maybe a better question to ask is: “if Bill Gates and Paul Allen were starting over today, what kind of a business would they start?”

My guess? They would sit around a dinner table and ask themselves:

What do eBay, Yahoo, Google, PayPal, iTunes, and Skype have in common?

What do you think?

Can't We All Get Along?

2005-04-11T09:31:00.000-07:00

This part is a true story:

Last night I was on an American Airlines flight from Newark to Los Angeles, and it was delayed for three hours because of a faulty altimeter. (An altimeter measures the plane's altitude.) Apparently, American did not have an altimeter in its Newark maintenace facility, so it had to have one driven from JFK, which is an hour away on a good day. Needless to say, yesterday was not a good day.

When we finally pulled into the gate at LAX around midnight, I went up to the pilot and spoke with him briefly. He confirmed what I had suspected all along:

The Boeing 757's altimeter is a standard part which is used by all the major airlines, including American, United, Delta, and Continental.
Since Newark is one of Continental's main hubs, there were probably several altimeters in the Continental maintenance facility.
If American had been able to borrow or buy an altimeter from Continental's facility, we would have been in the air with only a one hour delay. There are, however, no agreements between airlines for sharing parts and maintenance resources.

This part is my analysis:

It seems that if (1) and (2) above are true, then American and Continental could both benefit by sharing Continental's maintenance facility at Newark. American has a relatively small presence at Newark and could eliminate the high fixed costs of supporting its own maintenance facility for such a small presence. Continental, in turn, could earn a higher return on its larger maintenance facility at its Newark hub by charging a premium for these parts to the other airlines and improving parts turnover. And the 200 passengers on our flight last night would have been less irate, and the unfortunate flight crew would not have had to face their ire.

But I was not one of the irate passengers. First of all, I had a great weekend in New York and was glad to be coming home. Second of all, from what little I know, I thought we really did need the altimeter to fly. Finally, because I'm in the software industry, I realized that I should not be the one to cast a stone.

Think about it. The airline industry actually has standard parts such as an "altimeter." They have the technology standards required for interoperability. What they are lacking is the social organization for interoperability, in the form of maintenance facilities and inventories sharing agreements.

The lack of sharing agreements between airlines is a high hurdle to overcome, of course, but not nearly as high as those for interoperability in the software industry. In our industry, many participants continue to fight even common technological standards, trying to put up walls and moats around their own proprietary technologies and standards. Needless to say, a social organization or culture of sharing is much farther away still.

If we, as customers, would like that from vendors in other industries, why wouldn't our customers want the same thing from us? Don't our customers want the ability to select and use applications and components from a variety of vendors to meet their needs?

Are we going to be late getting them there as well?

Open Source Business Conference (OSBC)

2005-04-07T09:47:00.000-07:00

My friend told me that the first thing he looks for in an industry is a conference. "Once there is a conference, you know it's a real industry."

So it was with great excitement that I went to the second annual Open Source Business Conference. In the Gilded Age splendor of the Westin St. Francis, about 700 excited open source enthusiasts, curious to cautious to optimistic venture capitalists, hopeful entrepreneurs, sagacious lawyers, and fast-typing journalists mingled.

Here's my summary of the conference:

Open source is a legimitate business now. Few at the conference tried to justify that "free" can also mean "business." (In that sense, software has finally joined the rest of American industry, where businesses are built on things other than intellectual property secrets.)
The users, in this case CIO's of some very large organizations, were very excited about open source and simply wanted more of it--more projects, more support, more services to help identify projects. (Though, to be fair, they are a biased sampling of CIO's out there.)
Venture capitalists have seized on the fact that open source is a great distribution model and could potentially do away with the inefficiencies of sales and marketing in software. One statistic cited was that software licensing revenues are 76% of software sales & marketing revenues. Translation: users are paying vendors to sell to them. This is fundamentally inefficient, and they view open source as a direct, more efficient model of selling software.
Business models of services around open source are well established. (Curiously, in the Geoffrey Moore talk, the audience rated "open source service businesses" as more mature than any open source project such as Linux or MySQL.)
There was also active interest in deployment tools, management tools, devices, appliances, on-demand (hosting) businesses around open source software.
There was real disagreement over the future of open source in applications. One VC, the keynote speaker, was enthusiastic about it. The other VCs, in the panel discussion, were less so.
When a correspondent from The New York Times asked the VC panel why this time it would be different, one responded that the valley is a "manic-depressive" place. All of the VCs are publicly cautious about open source. One noted the open source is fundamentally "capital frugal, intellect intensive" (which I agree with very much.) At the same time, they were lots of VCs there talking to projects and entrepreneurs, so who knows?

I came back from the conference wanting more. It seems that the "open source" industry, if there is indeed such a thing, is still in the early stages, and people are still groping at the right business models. It's wonderful that services, appliances, deployment tools, etc. are well-established business concepts now. It's wonderful that VCs are willing to fund them.

But the conference largely skirted the ultimate power of the open source process: how will it fundamentally alter the dynamics--the economic relationships--in the software industry? In other words, not just sales and marketing, but also how software could be created and financed and how software companies can be organized and managed.

Could open source created a more frictionless model for the software industry? Instead of requiring huge investments to produce software and then hoping that users will buy them (and investing more money to convince people to buy them), could software be more of a demand-driven industry? A similar transformation is occurring in just about every other industry, from manufacturing to retailing to construction. Could it also happen in software? Could open source make it happen?

And then, from there--could open source unleash new creativity and help create whole new ways that software can benefit people and businesses? Software has become a living, breathing part of all our lives and all our businesses. Such creativity could surely be the greatest gift open source could bestow upon us all.

I think it could do both--create a more frictionless, demand-driven, "pulled" software industry and unleash new creativity in the industry. It will take new forms of not just software development but also organization of the industry. It will change how software companies are started, financed, and managed.

And that's the most exciting thing, as I look ahead to the future of open source. I hope to be part of inventing that future.

Open Source and the Road to Mainstream

2005-04-07T06:49:00.000-07:00

I keep coming back to the notion that open source is following its enabling technology, the Internet, from academia into the mainstream. The road to mainstream will, in my opinion, be similar as well. Specifically, it will resemble the road of the online retail industry.

In the beginning, of course, nobody believed that people would buy things online. Then startups such as Amazon.com showed that people, in fact, did buy things online, and eCommerce took off as its own industry. (Remember the days of the "Internet analyst"?) Next, established retailers responded by setting up separate online subsidiaries of their own, partly in hopes of cashing out in the dot-com boom, but also because they could not integrate online easily with their stores. (Remember when online purchases couldn't be returned in the stores?) Finally, eCommerce became mainstream, as startups such as Amazon.com have matured into multi-billion businesses, and established retailers such as Wal-Mart and Staples have fully embraced online as part of their core business. Whereas once there was an "eCommerce" industry, all retailers today speak of "multi-channel", "bricks and clicks," "clicks and mortar" strategies of integrated selling through online, catalog, and store channels. The reason? Simple--that's what the customers wanted: lower prices and convenience.

What is interesting is that eCommerce became mainstream not because the startups displaced the established retailers, but because established retailers have fully embraced eCommerce. Consumers shop online at amazon.com as well barnesandnoble.com, target.com, staples.com, etc. etc. In fact, the top retailers from an annual list compiled by Internet Retailer, the trade publication of eCommerce, is a who's who of major established retailers.

Open source software will follow a similar evolution. Originally, of course, nobody believed there could be businesses around open source. Now we are past that and into the second phase and third phase, where venture capitalists are funding open source startups and traditional software vendors are responding with open source projects of their own. The logical final phase would be traditional software vendors embracing the open source model as an integral part of their business, taking from and contributing to open source projects. Then open source will simply be mainstream. It's not clear when this will happen, but it is a matter of time. The reason? Again, simple--what the customers want: lower prices, better software.

One really interesting company to watch is Novell. Here is a billion dollar software company that is literally betting its future on open source. It is not just releasing existing software to open source, but rather adopting open source as the core of its future business. If Novell succeeds at bringing open source to its customers and building a business around services, it will send a powerful message to the entire industry. If it does not succeed, it will offer some equally powerful lessons on how the road into the mainstream should be traveled.

On a closing note, Microsoft has also joined the open source community. In addition to sponsoring open source conferences, it has actually created its first open source project Wix, which has gotten over 120,000 downloads to date. No company understands the business and strategies of software better than Microsoft. It will be interesting to see where they go from here.

The open source community should give them a big applause of welcome.

Open Source and Software Business Valuations

2005-04-01T13:21:00.000-08:00

Do Wall Street financial analysts understand open source?

The article Novell Gaffe Costly says:

"because license revenue is a key indicator of new business, while maintenance revenue reflects ongoing payment from current clients, the shift is disturbing."

It also specifically quotes an analyst as saying:

"the leading indicator of Novell's business -- software license revenue -- is caving in faster than previously thought, suggesting more severe market-share losses with NetWare and other products."

But how will licensing revenues continue to be the leading indicator of a software company's business under open source? More specifically, how will licensing revenues continue to be the leading indicator of Novell's business as it switches from proprietary to open source products?

Broadly speaking, software companies make R&D investments to develop products and then sell licenses to users, then follow through with services, support, and maintenance on those products. Licensing is thus a mechanism for a software company to recover its large investments in R&D. Software companies have huge operating leverage because the R&D investments are sunk costs bear no relationship to the number of licenses sold. In this way, developing software is rather like producing a movie. (Imagine that when you look at your programmer friends next time!)

Why do financial analysts focus on licensing revenues? Because, first of all, licensing sales have no marginal costs and thus huge gross margins. Second, and more importantly, licensing is a harbinger of the adoption base for the company's technology. Robust licensing sales indicate that the company's technology is gaining adoption. This adoption base determines whether the software company can derive future revenues from services or new products, and, critically, whether the company and its technology continue to remain relevant.

Of course, I'm sure there are some software companies where the only source of revenue is from licensing. I don't know much about this sector, but video games come to mind right now. For companies such as Novell, though, their products have significant future service revenue streams. Thus, for these companies, licensing is more important as a potential gauge of the adoption base for their technology.

Most open source companies, however, do not have licensing revenues. (Except for the few companies in the "dual licensing" business. My opinion on that subject does not belong here.) So right there, we need a different metric to measure the adoption base of their technology.

What about the high margins of licensing revenues? Does the absence of licensing necessarily imply that open source companies are in low margin businesses? Not necessarily either. First, because they can draw off a larger, open source code base, open source companies' investment requirements in R&D are also smaller. So, in a sense, they do not have to have licensing revenues to recover those investments.

Second, if their investments in open source community R&D help them build a stronger brand name and charge a premium price for services, support, and consulting, then I would argue they are earning very high returns on those R&D investments. If the adoption base of the underlying technology is large enough, they could potentially earn a far higher return on those R&D investments than most commercial software vendors ever could.

I hope this will convince some of my former Wall Street colleagues to sharpen their pencils.

The True Nature of Open Source

2005-03-30T13:00:00.000-08:00

Open source is frequently described as technology. I believe, however, that it is not a new technology itself, but rather a change in the social organization of the software industry.

Consider a historical example. Gunpowder, and its associated products, such as cannons and muskets, were new technological innovations. But the true impact of this new technology was in the organization of armies. Mass conscription infantrymen replaced the knights and elite calvary of the Middle Ages. Artillery rendered fortified cities with their massive walls and towers obsolete. Professional staff officers began to supplement, then replace, hereditary noblemen as military leaders. All of this social change was touched off by the introduction of new technology, in this case cannons and muskets, but the change itself was social and organizational.

Similarly, the open source movement was accelerated by new technology. In this case, the new technology is obviously the Internet, which becomes a vehicle for collaboration on and distribution of software. In the early days of Unix, UC Berkeley might send out a few hundred to a thousand copies on magnetic tape to other universities. Today, open source software is downloaded by the millions. Open source has followed the Internet out of the obscure corners of academia and into the mainstream software industry.

But open source itself is not a new technology. It is not a new methodology for creating software, the way that Unified Processes, UMLs, Agile or Extreme Programming are. It is also not a tool for creating software, the way that emacs, gcc, Eclipse, or Subversion are. It is certainly not a new programing language like Java, Python, or Ruby.

Instead, it is a new form of social organization. Users, producers, and distributors play different roles under open source models than under traditional software models. Different hierarchies exist in the open source markets. Old revenue models, namely licensing, no longer apply, but new ones are also being invented. That, I believe, is true excitement behind open source--the opportunity to create a new form of social and economic organization.