|
On this page:
Note: this article was originally written on request of Auke Jilderda as a testimonial of the Philips internal "innersource initiative" in March 2003. It is reworked in November 2005 to reflect the move to sourceforge.net.
This article describes the real-life experiences of the PFSPD project, showing a development from Philips internal closed source to a public open source methodology. PFSPD is an acronym of Philips File Standard for Pictoral Data. PFSPD is a file format to store video sequences. It originates from the end of the 1980s when it was known as PTSPD (Philips Tape Standard for Pictoral Data - storing usefull video sequences on harddisk was impossible at those days). The format was supported by a first generation library and tool set. This was maintained by an internal software development department; releases were only distributed in binary form within Philips. At the end of the 1990s, some Research employees felt the need for PFSPD support on different platforms with a very easy, straight forward access. This led to initial version of the cpfspd library. As this was a volutary effort, the software was distributed with source code within Philips. The project aims at a complete, platform independent development infrastructure with access library, graphics viewer and player and many conversion tools. As a non-official project, this development depends on contributions of volunteers, who work on the subject partly in Philips’ time and partly in their private time. In 2001, a Philips wide "innersource initiative" setup a sourceforge server at the Philips intranet. This fueled the developments further. Over the last couple of years four main developers (located in Europe and in the USA) worked on the project with contributions of 15 persons in total. The software has grown to more than 120,000 lines of C and C++ code. It has become the de-facto standard environment for research and development of video algorithms and implementations. In 2005 it was decided to bring the core library and viewer tools under the (L)GPL and release it to the general public. This tool set is aimed at video algorithm research and development activities.
PFSPD is a file format for video sequences. The file format has been developed early 1990’s. Also an access library (pfspd-al) and supporting tools (conversion, viewer, etc.) were developed, resulting in a UNIX/X based development environment. Apart from this software development environment, a high quality real-time video capture and display infrastructure exists within Philips Research to capture video signals or display them in real-time.
The file format is widely used by many research and development departments within Philips. Furthermore, an extensive set of video test sequences has been collected over the years, yielding a valuable asset of the company.
With the emerging importance of the PC platform in the engineering community and the appearance of the TriMedia processor for video signal processing, the need for PFSPD support on a wider range of platforms arose. The existing pfspd-al library shows signs of aging and was not readily available for the new platforms. We therefore decided to start writing our own simple access library containing just the few basic functions to suit our needs, aiming at ease of use and full portability... Robert Jan Schutten gave birth to the cpfspd library. The first routines were written around 1997/1998.
Soon we felt the need for a simple tool to generate test sequences. And a simple tool to compare two pfspd files in order to locate differences (and debug our algorithm software). And a simple tool to convert video data from one (pfspd) format into an other. And a tool to concatenate files. Bram Riemens created most of these initial tools.
As a next step, we felt that it would be great to be able to inspect video files on the PC platform. Thus Martijn van Balen started development of a simple tool to view PFSPD files. PV (pfspd viewer) was born.
From the very start, we aimed at full platform independence and high quality source code that is really portable and easy to use. The PFSPD file format was never changed to exploit the vast amount of video data that is already available. As result of our drive for ease of use, compatibility with the old pfspd-al API was not considered. However, the new API was designed to allow easy porting of applications from the old to the new api. All these aspects were explicit design choices; well thought when we started and never reconsidered.
This happened in the course of 1999 and 2000. This PFSPD activity has never been an official project, contributions were done by volunteers, parly in Philips’ time and partly in people’s private time (not to mention the programming efforts during long-and-boring flights over the Atlantic or during jet-lag hours when after giving up hope to get some sleep during darkness).
The library and tools were distributed to various product divisions simultaneous with our research results. Version control and configuration management was based on CVS. This widely available open source tool proved very stable and easy to use. As normal developer, only five simple commands are required! A project administrator uses a few more commands. Regular software releases were distributed by email to an increasing number of users. From the start, we distributed the source code together with precompiled executables. This relieved support effort: the exec’s work on all our own systems (HP, Sun, SGI, Linux, Windows), other users (at other sites) can compile in their own environment.
In the beginning of 2001, we heard about Auke Jilderda’s "innersource" initiative via personal contacts. This initiative aimes at promoting the principles of the open source community as collaboration and code sharing for company wide use. Soon we decided to create one of the first projects on the newly installed (philips internal) SourceForge server. We setup the CVS repository on the server and transferred our software to this new repository. Since we were used to CVS, normal development was hardly affected (the behavior of CVS commands remains identical whether it access the repository on a shared disk or whether it accesses the repository over the network on the SourceForge server). Main reasons to move to SourceForge were:
Main features were therefore the version control system and release download pages. Most important advantage was the reduction of email handling time (no software distribution via email anymore, but the website also avoided email questions).
In the summer of 2001, Robert Jan Schutten moved to Philips Semiconductors in Sunnyvale, CA, USA. Suddenly, the server-client nature of the SourceForge system became a huge advantage, since Robert Jan remained part of the development team. At the end of 2001, there was a need for fast development of a "natural motion" function on TriMedia using a new hardware platform. In that period Robert Jan and Bram worked together on a port to this new platform, integrating a new board support pack. For more than two weeks, we worked almost around the clock, effectively taking advantage of the 9 hour time difference between Eindhoven and Sunnyvale. By communicating code changes via the CVS repository, one could "catch up" with each other’s modifications in a minute and continue working on a new feature. Exciting!
As Auke Jilderda and the system management team extended the (philips internal) SourceForge service with mailing lists, we were again amongst the early adopters. A (restricted) developers mailing list is used extensively to discuss design issues, testing strategies, bugs and (most importantly) report day-to-day progress to the development team. Using this mailing list, the team is informed about (the reason for) all modifications in the code base. On busy times, this mailing list carries more than 5 emails a day (which is one of the reasons to have it restricted: it is too detailed and too disturbing for most users). A public users mailing list is setup to aid in creation of a PFSPD users community. This mailing list is intended to discuss future directions and new requirements, obtain help and mutual assistance, inform the community about new releases, etc. This users mailing list is growing in popularity (currently 24 subscribers) though it could be better integrated in the day-to-day development culture.
Developments on many tools continued steadily, increasing the number of PFSPD utilities to 11 and providing full support of all features in the PFSPD format. In the beginning of 2002, the cpfspd library was significantly enhanced to support large files (> 2 Gbyte) on all systems. Furthermore it now supports real-time performance on the Windows platform (NTFS file system). All these extensions were added without affecting the existing API.
In the summer of 2002, the project entered a new phase. Kees van Zon from Philips Research Briarcliff (NY, USA) was added to the development team. He proposed "pts" (pfspd tool shop), a Philips propriarity tool with advanced video signal processing functions). After initial discussion, we decided to jointly build this tool and integrate lots of existing functionality into pts. Bram set up this new development tree and ported the existing software to the new environment. Kees added a significant amount of new functionality. Again, remote cooperation proved very valuable and SourgeForge support was indispensable.
Here are Kees' experiences with joining the PDSPD development team:
After conceiving the idea of "pts", I initially wrote a version that was independent of the PFSPD software on SourceForge. I was familiar with some of its concepts though, such as the Make Took Kit and the command line parser, which I both adopted simply because they are very good. At some point I took a closer look on what PFSPD tools were actually available via SourceForge, and saw many that would be great additions to "pts". I sent Bram the preliminary version that I had written, and after some discussion back and forth, we decided to join forces and combine all functionality suitable for the new concept into one piece of code that would be offered via SourceForge. I became a member of the PFSPD development team, and adopted the use of CVS via SourceForge. Thanks to excellent on-line documentation, including the project philosophy, a developers guide, and a CVS guide, this went quite smoothly. Most of "pts" was developed in two busy months that followed. The developers mailing list proved an excellent means of communication. Responses were always quick, and the time difference hardly restricted the speed of development.
In summary, SourceForge proved to be an excellent platform for multi-site software development. From my perspective as a newcomer to the PFSPD development team, the keys to success are i) a well thought-out way of working (e.g. CVS, directory structure); ii) good on-line documentation; iii) the developers mailing list; iv) an active development community where issues get addressed quickly; v) last not but least, a knowledgeable champion that actively initiates, stimulates and guides the activity.
The cpfspd library has become the de-facto standard in the video signal processing community within Philips due to its ease of use and widespread availability.
PV (pfspd viewer) and PP (pfspd player) have become the dominant pfspd viewer resp. player.
The project now consists of about 37,000 line of C and C++ code (not counting pts). 4 persons developed most of this code, which also contains contributions of a dozen others.
Apart from contributions by the developer team, this project has also catalyzed various other developments. First of all, existing software of other people was integrated, increasing the ease of use and distribution/visibility. The cpfspd library has been the key enabler to accept the PC platform as development environment for video algorithms and implementations. This enabled new pfspd applications on this (relatively) cheap platform. It further spawned other initiatives, like:
The development of these PFSPD related software packages would have been impossible without the SourceForge support. It has proven to be a great tool to enhance productivity, especially when cooperating between multiple PD’s and over large geographical distances.
On the other hand, communication via computer means is very limited compared to face-to-face discussions. We were only able to cooperate over these large distances because we had developed a common design approach, a common way of working and a common attitude towards code style, testing and quality assurance. The importance of these aspects can hardly be overestimated! Therefore, these issues have to be made explicit, so we decided to document these aspects (at least the most important ones). We used the PFSPD website for this, which improves transparency of the development and testing processes and aids in managing the expectance level of the potential users.
Innersource development proved a huge advantage. One interesting characteristic about innersource is the decreased dependency of users on the original developers. We welcome that; as developers we do n’ot have a desire to "bind" our users (we have no commercial interest, nor any desire for support, etc). Furthermore, we experienced that the availability of the source code lowers the threshold of using the software because people feel to be "in control". This seems particularly important in research or advanced development communities were people work in a very explorative way. Up till now, we have hardly seen derived software versions, which is probably caused by our own pro-active attitude and drive to make high-quality software that is well suited for its goals.
Although it seems obvious, we have taken very much care to give credits to the people who deserve it. Since this effort is solely based on volunteering contributions of a few developers and "donations" by other colleagues, these people receive no other reward than the honor to participate and the personal satisfaction to work on a nice and valuable piece of software. We have learned the importance of this issue the hard way: once we used someone’s code and distributing it without consensus with the original author, unintentionally leaving him with the feeling that we had "stolen" his code. It took a while before this was straightened out...
Creating a user community proved more difficult than we expected. It seems that people more easily email or phone one of the developers than sending an email to a (more anonymous) mailing list. In cases where the mailing list is used, we have seen very adequate responses from unexpected corners though! We therefore continue our efforts to encourage people to use the mailing list.
Finally, it is no surprise that the development of this PFSPD system took a considerable effort. Since it can be used freely within Philips, we have no insight in the number of users or business impact of the software. As developer community, we welcome that since it alleviates us from support tasks. From management point of view it seems to contradict with current trends to increasingly measure the performance of an organization. So the innersource and open distribution character that lead to easy adoption of the software and thus to its success, complicates justification of the development efforts. We need to learn more about how to apply such open development processes in a business environment.
This project is driven by the creativity and quality drive of a few individuals. The SourceForge server provided the infrastructure to materialize this drive into a comprehensive tool set that has become the de-facto standard in Philips.
In 2005, it became appearent that it could be beneficial to Philips to open the pfspd format to the world. After internal discussions and a formal decision process, the following was decided:
Authors:
Bram Riemens
Kees van Zon
Special thanks to Ralph Braspenning and Auke Jilderda for their
contributions and constructive review.
Copyright (c) KONINKLIJKE PHILIPS ELECTRONICS N.V. 2000-2006
Philips Research Laboratories, Eindhoven, The Netherlands
CVS id: $Id: history.html,v 1.5 2006/07/04 08:04:21 riemens Exp $