Nov 27, 2008

Digital growth for Warner Music & Atlantic records

Key figures for Warner in 2008:
  • Global revenue: +3% ($3.49 billion)
  • Global digital revenue: +39% ($639 million), 18% of total revenue
Atlantic Records, a label from the Warner Music Group, claims that digital sales now account for 51% of its U.S. revenue, the first music label to pass the 50% mark.

Nov 24, 2008

Yngwie comes to Rock Band!

Best news for at least 6 months : 3 songs from Yngwie Malmsteen's latest album will soon be downloadable for Rock Band. Warm up those fingers and prepare for pain.

Yngwie is God (2,388 listens can't be wrong) and now I *have* to buy Rock Band. Oh well... ;)

EU Culture Ministers on legal offers, DRM and privacy

The Council of EU Culture Ministers met on November 20 to discuss the "development of legal offers of online cultural and creative content and the prevention and combating of piracy in the digital environment"

Here are some interesting excepts from their conclusions :

"Although the use of technical systems for protection (Technical Protection Measures (TPM)) or rights-management information (Digital Rights Management (DRM)) may in some cases, while taking due account of the principle of personal-data protection, contribute to protecting and managing rights in the digital world, the lack of interoperability or transparency of these systems creates insecurity for consumers and limits their use of the content offered by the various platforms".

"[Should be taken into account] the need to ensure a fair balance between the various fundamental rights, particularly the right to personal data protection, freedom of expression and information and the protection of intellectual property, and to seek, when implementing Community law, solutions in compliance with the general principles of Community law, in particular the principle of proportionality"

Does this mean that Amendment 138 will not be attacked by the Council of EU Telecom Ministers on November 27? Who knows...

Stay tuned.

Nov 20, 2008 vikings axe WMA

Following in the footsteps of US behemoth Walmart, the largest E-retailer in Scandinavia - - has decided to drop the DRM-ridden Windows Media Audio (WMA) format for digital music downloads.

Indeed, CDON announced today that the 2.5 million tracks present on their Finnish store will be available in the MP3 format. Says CDON CEO Mikael Olander: "it's only with legal workable alternatives we can prohibit illegal downloads and so far the music industry have been way to slow".

Vikings never disappoint. Well done.

More DRM evilness... by Apple this time

As you probably know, the only external monitor connection available on the latest Mac Books is a DisplayPort connection.

As you probably don't know (I didn't), DisplayPort implements a copy protection scheme, called DisplayPort Content Protection (DPCP), which prevents copyrighted content from being viewed on non DPCP-compliant screens.

This gibberish means that many owners of new Mac Books can't use their not-so-old VGA/DVI monitor to view copyrighted content they have paid for. I'm sure that pirated content plays fine, though...

Read the full story.

Nov 19, 2008

MySQL conference in Paris - High-availability with MySQL

Here's another good talk from the MySQL conference in Paris today: High-availability with MySQL by Max Mether, Sun Microsystems

Again, here are my raw notes.

MySQL replication
  • Pros
    • Simple to use and maintain
    • Support for "active-active'" configuration, but without conflict resolution --> logical separation required to prevent them
  • Cons
    • Asynchronous
    • No automatic failover (heartbeat must be installed to support HA)
    • No default monitoring
    • slave is mono-thread --> slave can lag behind on multiple transactional updates to the same table
  • Binlog files : change requests (up to 5.1), change images (since 5.1)
    • every change to the binlog is sent to the slave, which writes it to the relay log (I/O thread)
    • every change to the relay log is executed by the slave (SQL thread)
  • Distributed Replicated Block Device
  • Synchronous replication :
    • I/O operations are sent via TCP/IP to the slave, which commits them to disk and confirms the operation.
    • Active / passive configuration
  • Works great with Linux HA & heartbeat : failover complete in about 30 seconds
  • Pros
    • Simple
    • Synchronous
    • Integration with Linux HA
  • Cons : small performance impact
  • Continuent :
    • Active / active configuration
    • Written in Java
    • The UniCluster controller replicates data between both nodes
    • Support for Oracle / MySQL replication
Shared storage
  • Active / Passive configuration
  • Automatic management
    • Virtual IPs
    • Failover
    • Data synchronization
    • File system management
  • Pros
    • Good performance
  • Cons
    • Storage can be a SPOF
    • Costly NAS/SAN storage more expensive than DAS
    • Unused resources
    • Failover time is higher
MySQL cluster
  • "Shared-nothing" architecture (no SPOF)
  • Technology bought from Ericsson in 2003 (initially developed for telcos)
  • Three node types
    • Data nodes (aka MySQL servers)
      • Store and manage data
      • Automative failover
      • Automatic synchronous replication within a node group
      • Data can be partitioned between node groups
      • All nodes can be active
    • SQL nodes
      • SQL processing
      • Native API
    • Management nodes
  • Cons
    • multiple MySQL servers required (3 minimum, 5 recommended)
    • SQL processing not always optimized for clustering

MySQL conference in Paris - MySQL roadmap

I attended the MySQL conference in Paris today.

As always, presentations are kind of hit & miss.
The best one was by far the MySQL Roadmap by Robin Schumacher, Director of Product Management at MySQL. Here's my (very raw) transcript.

MySQL Server 5.1
  • Faster : 11-18% over 5.0 (internal MySQL QA tests)
  • New features ( list @
    • Table/index partitioning : range, hash, key, list, composite
      • Partition pruning : only needed partitions are
    • Replication : statement-based, row-based, hybrid
      • Not only statement based replication (ex: random
        number generation)
      • Actual replication on row data
    • Built-in scheduler
    • Identification of SQL problems
      • Query logs can be activated dynamically (no restart required)
      • Query tuning
  • GA scheduled for CY Q42008
MySQL Server 6.0
  • Falcon engine : transactional engine
    • Not an InnoDB replacement (i.e. configuration files are different)
    • ACID transactions
    • Crash recovery
    • Record-based cache, not page-based
  • New backup features
    • SQL-command driver interface
    • Online, non-blocking for DML operations
    • Native driver --> faster backup (about 75%
      compared to mysqldump, about 90% for recovery)
    • All general engines supportes, except cluster
  • Optimizer enhancements
  • Replication
    • Semi-sync replication
    • Ensure commited slave
    • Replication heartbeat
    MySQL Entreprise
    • memcached part of MySQLEnterprise
    • MySQL Workbench : data modeling / database generation tool
    • zmanda : backup tool
    • MySQL query analyzer
      • Trace, monitor and analyze MySQL query activity
      • "Query sniffer" on the network
      • Analysis across all servers
      • Available now
    • MySQL load balancer
      • Q1/Q2 2009
    • MySQL connection manager
      • Connection concentrator ("multiplexer")
      • Connection pooler
      • Q1/Q2 2009
    • Carrier-grade edition compared to enterprise-edition
      • Geographical replication
      • LDAP interface
    MySQL 6.x
    • 6.1 :
      • Replication fail-over
      • Crash-safe slaves
      • Event checksums
    • 6.2 :
      • Multi-source replication
    • Extended support of foreign keys
    • GA Q4 2010
    MySQL 7 "Citadel"
    • Security groups / roles
    • Transparent data encryption
    • Data auditing (for compliance)
    • External authentication
    • Online "Alter Table"

    Nov 18, 2008

    Slides from last week's Spring conference in Paris

    The slides have been posted on the "Rencontres Spring" website:

    [Updated on 2009/01/09] Videos are available here.

    EMI Music top execs interviewed by Billboard

    Interesting interview of Elio Leoni-Sceti (CEO), Nick Gatfield (president of A&R for North America and the UK) and Douglas Merrill (worldwide president of digital, former Google CIO).

    Some highlights:

    Leoni-Sceti : "This industry has always relied on somebody else to innovate in the way that this content is delivered to music fans. Somebody has to invent the iPod, somebody has to invent the music experience and the various incarnations of this music experience. So we have been great at providing the content but somebody else had to take this content and deliver it in your way. We want to regain ownership of the innovation that goes with our product delivery."

    Merrill : "DRM certainly doesn't provide value to the fan (...) The pirates are doing a busy business regardless. The best way to get a pirated copy probably isn't to buy it from iTunes and then push it. We didn't see the needle move at all on [piracy]. But what we did see is consumers loved the product. It was good for consumers, it's good for artists. It gets people engaged with the art in a whole new way by getting rid of artificial rules like "we don't trust you, so I'm not going to give you this content". It just sort of set the wrong tone with our customers".

    Leoni-Sceti : "I think that "middleman" is a bad description of what our company and our industry should be. If you're a middleman and you're just shuffling the paper from one side to the other one, there's no role whatsoever to being there. If you are actually adding knowledge and enabling that relationship to occur, fans and artists need the industry more than any other time before".

    Nov 13, 2008

    Spring conference in Paris, part 3

    [Updated on 2009/01/09] Videos are available here.

    Part 1 & Part 2 already posted. This is the last one.

    Here's the transcript of Jurgen Hoeller's presentation on Spring 3.0.

    Spring 3.0 themes

    • Java 5+ foundation, compatible with J2EE 1.4 and Java EE 5
    • Spring Expression Language : Unified EL++
    • Comprehensive REST support
    • Declarative model validation : Hibernate Validator, JSR 303
    • Portlet 2.0
    Expression Language in bean definitions : dynamic expressions for property values
    <bean... >
    <property name = “databaseName” value=”#{systemProperties.databaseName}” />
    <property name = “keyGenerator” value=”#{strategyBean.databaseKeyGenerator}” />

    Expression Language in component annotation : dynamic expressions for method parameters
    public class RewardsTestDatabase {

    public void setDatabaseName(String dbName) {... }

    Conversation management
    • Key problem : isolating concurrent windows in the same browser
      • Shared HTTP session
      • Several independend conversations going on
    • Define scope with a shorter lifetime than session
      • Scope = “conversation”
      • Spring Web Flow will provide sophisticated flow
        navigation management on top of this
    REST support : Spring MVC will provide 1st class support for REST
    @RequestMapping(value=”/show/{id}”, method=GET)
    public Reward show(@UriVariable(“id”) long id) {
    return this.rewardsAdminService.findReward(id);

    Support for different representations
    • JSON : GET
    • XML : GET
    • ATOM : GET
    Model validation
    public class Reward {
    private Date transactionDate;
    • In view : <form:input path=”transactionDate”>
    • Description of constraints in the application itself
    • Same metadata can be used for persisting, rendering, etc.
    • Support for Hibernate Validator and Bean Validation (JSR 303)
    Spring 2.5 mission continued
    • Fully embracing Java 5
    • Backward compatibility with Spring 2.5
      • 100% compatibility of programming model
      • 95% compatibility of extension points
    • All previously deprecated API to be removed
      • code still working
      • documentation already removed
      • stop using 1.2 / 2.0 API !
    • 3.0 Milestone 1 : end of november
      • new build system
      • first cut of REST and Expression Language
    • 3.0 RC1 : march 2009
      • followed by 2 further milestones
    • 3.0 Final : april 2009
      • depending on RC feedback...

    Spring conference in Paris, part 2

    [Updated on 2009/01/09] Videos are available here.

    Part 1 here.

    Here's the transcript of Mark Thomas' presentation on Tomcat optimization & performance tuning.

    The process
    • Understand the system architecture
    • Stabilize the system (no one should be messing with it while investigation is in progress)
    • Set performance targets
      • All requests under 3 seconds ?
      • This type of request under 3 seconds, this one under 10 ?
      • 90% of requests under 3 seconds, 100% under 10 ?
    • Measure current performance
    • Identify the current bottleneck
    • Fix the root cause of the bottleneck, not the symptom, e.g. if your systems runs out of memory, don't add extra memory : find where the memory consumption occurs
    • Repeat the whole process until you meet the performance target
    Common errors
    • Optimizing code that doesn't need it
    • Insufficient testing
      • realistic data volumes
      • realistic user load
    • Lack of clear performance targets
    • Guessing where the bottleneck is
    • Fixing the symptom rather than the cause
    Tuning options
    • Applications typically account for over 80% of a request processing time, so look at your application first.
    • Tomcat logs are also a good place to start, as their default configuration is too generic
      • catch-all logger writes both to file and to stdout... but Linux redirects stdout to a file.
      • The catch-all file has no overflow protection. It will just grow and grow.
      • Logging is synchronous, which shouldn't be a problem for local disks but it does add overhead if logs are written through the network (NAS).
      • Solutions :
        • Remove logging to stdout, i.e. remove java.util.logging.ConsoleHandler from the handlers list in the file
        • Add log rotation to the catch-all logger
        • Asynchronous logging has not been implemented yet...
    • First, you need to understand your application usage patterns
      • One request every now and then ?
      • Short bursts of requests ?
      • One request every 3 seconds ?
    • TCP/HTTP/SSL connections
      • TCP connection setup is expensive, especially over WANs with high latency
        • HTTP keep-alives allow a TCP connection to be kept open and reused for other HTTP requests
      • SSL connection setup is very expensive
        • HTTP keep-alives are a must if SSL is heavily used
    • 3 connectors are available
      • Java Blocking I/O (BIO)
        • oldest one
        • most stable
        • JSSE-based SSL implementation : very slow
      • Java Non-Blocking I/O
        • JSSE-based SSL implementation : very slow
      • Native (APR)
    • Picking the right connector for a given requirement (from best to worst)
      • Stability : BIO, APR, NIO
      • SSL : APR, NIO, BIO
      • Low concurrency : BIO, APR, NIO
      • High concurrency, no keep-alives : BIO, APR, NIO
      • High concurrency, keep-alives : APR, NIO, BIO
    • Why use NIO at all ? It never comes first !
      • APR is unstable on Solaris
      • NIO is a pure Java solution
      • Switching between BIO and NIO is straightforward
        • Same configuration files
        • Same certificates
    • maxThreads
      • Maximum number of threads servicing HTTP requests
        • For BIO, this value really is the maximum number of concurrent client requests
      • Typical value : 200-800
      • 400 is a good starting point, which may be adjusted depending on CPU load
    • maxKeepAliveRequests
      • Maximum number of concurrent HTTP requests per TCP connection
        • 1 : no keep-alives
        • Typical value is 100
    • connectionTimeout
      • Typical value : 3000 ms
      • Also used for keep-alive timeout
      • Increase for :
        • slow clients, such as mobile phones
        • layer 7 load balancer with keep-alives
      • Decrease for faster timeouts (!)
    • Content cache
      • element
        • cacheMaxSize : typical value is 10240 Kb
        • cacheTTL : typical value is 5 s
      • NIO/APR can use sendfile for large static files
        • OS “bypass”
        • File sent by a different thread, which doesn't tie up a HTTP request processing thread
    JVM tuning
    • Memory
      • Xms / Xmx flags
        • Used to define the size of the Java heap
        • Aim to set as low as possible
        • Setting the heap too high wastes memory and can cause long GC pauses
      • XX:NewSize / XX:NewRatio
        • Set to 25/33% of total Java heap
        • Setting the ratio too high/low leads to inefficient GC
    • Garbage collection
      • GC pauses the application
      • From milliseconds to seconds
      • XX:MaxGCPauseMillis / XX:MaxGCMinorPauseMillis
        • these set goals, which are not guaranteed
        • they lead to more frequent, shorter pauses
    • More on
    Load balancing
    • Basic configuration : 1 httpd, 2 Tomcat instances, mod-proxy-http or mod-jk
    • Stateless requests
      • they are routed purely according to load balancing algorithm
      • this doesn't allow HTTP sessions
    • HTTP sessions
      • sticky sessions must be setup, i.e. all HTTP requests from a single session must go to the same Tomcat instance
      • this is performed by appending a session cookie to the HTTP requests, which will be tracked by the load balancer
    • Session replication between Tomcat instances can be added through clustering
    • Replication is asynchronous by default (done once the answer has been sent back to the client
    • Single line configuration by default.
    • Additional configuration needed for real-life production
      • Automatic discovery of new Tomcat instances (through IP multicast)
      • Synchronous replication (watch out for the performance impact)
      • Session replication to a specific node (instead of all nodes)
    Hints & tips
    • Use a minimum of 3 Tomcat instances
    • Test your application with load balancing and clustering before going to production (it may not behave the same)
    • Redeployment can cause memory leaks
      • Include it in your testing
      • Safer option : do a stop/start upgrade on each individual Tomcat instance in your clustering

    Spring conference in Paris, part 1

    [Updated on 2009/01/09] Videos are available here.

    Just got back from the Rencontres Spring. Here's the transcript of Peter Cooper-Ellis' presentation. He's Senior VP Products at SpringSource and a co-founder of the company.

    Additional transcripts to follow (Mark Thomas on Tomcat tuning and Juergen Hoeller on Spring 3.0)

    Solve the complexity crisis
    • Provide “weapons for the war on complexity” (Rod Johnson)
      • power through simplicity
      • flexible, not monolithic
      • continuity, current & future
    • Improve application quality and staff productivity
    • Deliver excellence through open source & standards
    • Lead technically within the Spring Framework, Apache
      Software Foundation, Eclipse Foundation and Java Community Process
    Develop, Deploy, Support
    • Develop
      • SpringSource Tool Suite
      • Spring community project
      • Spring programming model
    • Deploy
      • SpringSource Enterprise
      • SpringSource ERS
      • SpringSource dmServer
      • Performance packs
    • Support
      • SpringSource Application Management Suite (AMS)
      • Development & production subscriptions
      • Consulting & Training
    Programming Model Roadmap
    • Q4 CY08
      • Spring .NET 1.2 : quick start template, ASP .NET MVC
      • Spring Integration 1.0 (“pratical approach to an enterprise service bus”)
        • routing & transformation of messages
        • Asynchronous, loosely-coupled components
        • Configuration
    • Q1 CY09
      • Spring 3.0
        • REST
        • Java 5
        • Expression Language
      • Spring DM 1.2
      • Spring IDE 2.5
      • Groovy 1.6
      • Grails 1.1
    • Q2 CY09
      • Spring DM 2.0 : blueprint service
      • Spring Batch 2.0
        • Java 5 and Spring 3
        • Non-sequential execution
        • Scalability
        • Configuration : annotations & namespaces
      • Spring ROO : “dynamic language”
        • Convention over configuration
        • Rapid application development
        • DDD
      • Spring Security 2.5
        • Documentation upgrade
      • Spring Web
        • Flex
        • Dev kit : scaffolding & patterns
        • Declarative validation
    SpringSource Enterprise
    • Complete portability over runtime environments
    • Package of open-source applications and commercials add-ons
      (monitoring, performance, ...)
    • SpringSource Tool Suite : Spring IDE + Eclipse Mylyn + integrated SpringSource Knowledge Base
    • SpringSource Application Management Suite
      • Monitor production deployments of Spring applications
        • Tomcat, Weblogic, Websphere, Jboss
      • Track application performance, utilization & resource control
        • Monitor servers, application contexts & bean factories
      • Define critical application alerts with automated corrective controls
      • Control and record changes
    • SpringSource Advanced Pack for Oracle Database
      • Oracle RAC fast connection failover
      • Oracle Streams advanced queuing
    SpringSource dmServer
    • Next-gen, completely module-based application server
    • Runs on the SpringSource dm Kernel
    • Harnesses the power of Spring, Tomcat & OSGi
    • Roadmap
      • Sep 08 : dmServer 1.0
      • Q2 CY09 : dmServer "Jersey"
        • Distributed management
        • Management architecture
        • Compatibility with Tomcat & Spring
      • Q3 CY09 : dmServer "Orkney"
        • Servic Fabric
        • Dynamic load balancing
        • Dynamic provisioning
        • Payload aware routing
        • Management
          • rolling upgrade of applications
          • rolling upgrade of servers
          • record & playback script of console session

    Nov 12, 2008

    Sun back in the storage game ?

    Sun Microsystems has announced their new storage line, which they call "Sun Storage 7000 Unified Storage Systems". These systems are based on x64 architectures (AMD dual or quad-core), 1 Tb SATA disks, SSD add-ons for caching and of course ZFS.

    The high-end system - the 7410 - supports a maximum of 288 Tb (raw), requires 48 rack units and about 7.5kW at max load.

    With the "clustering" option, a second system can be connected in order to provide redundancy and an additional 288 Tb. So what you'd get would be a NetApp-like configuration with dual controllers and a total of 576 Tb.

    According to the Sun website, this clustered configuration's list price would be close to a million dollars, which translates into $1750 per raw terabyte. Although this still isn't a true cluster (in an Isilon way), ZFS *is* very cool and with a nice discount, this could be worth a look...

    Nov 10, 2008

    EMC: Maui becomes Atmos

    After months of delay and speculation, details finally emerge on EMC's cloud storage technology.

    Initially code-named "Maui", the software part of the product is now called "Atmos". Some sort of oh-so-clever pun on clouds, I suppose. Says EMC: "Atmos is a multi-petabyte platform for information storage and distribution". Hmm, ok. This does look like Berkeley's Oceanstore... maybe because M. Eaton, one of Atmos' architects, was also a key contributor to Oceanstore :)

    Atmos can either run on EMC hardware (which looks like x86 servers and SATA disks), or on your own hardware, i.e. x86 servers and SATA disks... but probably quite less expensive :))

    OK, that's it, then: managing petabytes of data spread over distributed islands of commodity storage. No pricing information for the moment. Wait and see...

    SACEM chairman hates everyone

    The SACEM is a French professional association collecting payments of artists’ rights and redistributing them to original authors, composers and publishers.

    Their president, M. Petitgirard, has recently given a couple of interviews to LCI (French 24/7 news TV), which I think deserve to be... highlighted:
    • "If the law doesn't pass [HAPODI law, infamous for the "three strikes" policy], we will launch thousands of procedures. Tens of thousands if needed".
    • "Maybe iTunes makes a little money, but what they want is simply to sell iPods : they are hardware vendors. It's an alibi".
    • He adds that "Apple loves money a lot... a music a little bit" and also that "music majors are not necessarily my friends, but they are facing a vendor who makes a mockery of authors, performers and producers".
    • To top it all, he calls ISPs "vampires" who "live off the illicit exchange of cultural goods".
    Whatever this guy is on, he needs to lay it off now and take a long, hard look at R-E-A-L-I-T-Y.

    Digital distribution is challenging the existence of collecting societies ("rightfully so", many people would say). Artists and producers are starting to realize thay, hey, maybe they don't need them anymore. As their cake gets smaller, it's quite obvious that they will be taking a good look at who's eating how much... and why.

    In fact, hardly anyone wants collecting societies to survive, except maybe the top 20 French artists who get a lot of money from them. Even the EU wants to break their national monopoly.

    SACEM and the likes are dinosaurs facing extinction. Adapt or die.

    PS: ah, there *is* a tiny of bit useful information in these interviews. Deezer, the famous streaming music website, is paying SACEM 0.007€ per listen.

    Amendment 138 accepted by European Commission

    The now famous Amendment 138 - which directly opposes the so-called "three strikes" policy implemented by the French Government in the HADOPI law project - is still on, says the European Commission.

    "Amendment 138 adopted by Parliament in its plenary vote on 24 September provides that "no restriction may be imposed on the fundamental rights and freedoms of end-users, without a prior ruling by the judicial authorities, notably in accordance with Article 11 of the Charter of Fundamental Rights of the European Union on freedom of expression and information, save when public security is threatened where the ruling may be subsequent." As already stated on 6 October, the Commission can accept this amendment, which was voted by a nine-tenths majority in the European Parliament plenary (MEMO/08/681). The Commission considers this amendment to be an important restatement of key legal principles of the Community legal order, especially of citizens' fundamental rights. It leaves Member States sufficient scope for reaching a fair balance between different fundamental rights, in particular the right to respect for private life, the right to protection of property, the right to an effective remedy and the right to freedom of expression and information."

    This message doesn't seem the echo in Paris... Having failed to kill the Amendment, the French Government stuck their head in the sand and rushed the HADOPI law in the French Senate, who approved it with an almost unanimous vote.

    Next episodes:
    • Nov 27 2008: council of EU Telecoms Ministers on November 27.
    • Q1 2009: second reading of the HADOPI law in the French Parliament scheduled for early 2009.
    • Apr 2009: second reading of the Telecom Package in the European Parliament.
    • 2010: the new regulatory framework is expected to become law in all 27 EU Member States.
    This is far, far from over. Get informed!

    Nov 7, 2008

    Nov 3, 2008

    OWASP NYC 2008 Conference

    The Open Web Application Security Project (OWASP) is a worldwide free and open community focused on improving the security of application software.

    All presentations from their last conference in NYC are available (slides, video or both) on their website. Lots of interesting stuff, such as:
    • Security of Software-as-a-Service (SaaS) - VIDEO
    • Security in Agile Development - VIDEO / SLIDES!!! :)