Even though MySQL is used to power a lot of web sites and applications that handle large binary objects (BLOBs) like images, videos or audio files, these objects are usually not stored in MySQL tables directly today. The reason for that is that the MySQL Client/Server protocol applies certain restrictions on the size of objects that can be returned and that the overall performance is not acceptable, as the current MySQL storage engines have not really been optimized to properly handle large numbers of BLOBs. To work around these limitations, these projects usually just store a reference to the object (e.g. a path name in a regular file system). This approach works around the limitations applied by the MySQL Server, but results in a disconnection and potential source of inconsistency between the database and the file system content. There was an interesting discussion about that topic on
Sheeri's blog some time ago, with an
excellent reply by Kristian, if you are curious to read more about this.
Thanks to the pluggable storage engine architecture of MySQL 5.1, we now see many independent storage engines emerging, one of them being PBXT by Paul McCullagh from SNAP Innovation. One of its strengths is going to be the handling of large binary objects in a performant way, so data objects can be stored in the database along with the other information related to it. To compliment this feature and to work around the limitations applied by the Client/Server Protocol, SNAP has now launched the Scalable BLOB Streaming Architecture for MySQL project, which intends to develop an architecture that accompanies the MySQL Client/Server protocol by providing an API to access and stream BLOBs stored in a MySQL database. See their press release (PDF) for the official announcement..
Paul will hold a BOF about this at the MySQL Conference and Expo this Tuesday at 19:30 in room San Thomas. So if storing and streaming large binary objects out of a MySQL Database is an essential requirement for your project, make sure to attend this session and provide Paul with your feedback and ideas!
