concena.com

Looking for Contract Nuxeo Developers

2013-04-12T16:11:14Z

We are currently looking for a few contract Nuxeo DM and DAM developers to help out on a number of client and internal projects. Experience with IDE-based Nuxeo plugin development is a must. Experience with Google Web Toolkit would be beneficial but is not required. If you are interested, please submit your resume, rate expectations and contact information to bruce@concena.com.

Great resource for Nuxeo developers...

2013-03-13T14:11:38Z

Whether you are starting out as a Nuxeo Developer or been working at it for years, the Nuxeo For Developers page is a great launch pad. Heck, even if you're not a Nuxeo developer, the resources for Maven and Git usage serve as great topical examples!

Comment on Nuxeo Tech Report 4 - DAM

2013-03-11T14:59:22Z

Studio and UI configurability enhancements in Nuxeo DAM are welcome improvements. Making more of the interface more easily configurable with help more quickly meet customer requirements.

Unfortunately, I think Nuxeo has missed the mark when it comes to DAM performance. While DM has certain performance requirements, they pale in comparison to DAM. The average Nuxeo DM document (with attachments) is orders of magnitude smaller than a typically DAM asset. In this case I am talking about the use of DAM for high resolution images, graphics, and video.

There are challenges storing, rendering, and presenting all of these large files. Hardware (RAM, disk speed, CPU) is an extremely important part of the equation, but equally so are the underlying libraries that extract metadata, render lower resolution images, and stream video. Nuxeo's own code, wrapped around these libraries is also an important (and sometimes limiting) factor.

I believe that for Nuxeo DAM to be useful (in a marketing firm for example) there needs to be significant work on performance and scalability of DAM.

That's my two cents, albeit based though on real life experience with DAM. Anybody else have similar experience with DAM and handling of large digital assets?

Integrating GWT Applications with Nuxeo

2013-03-02T17:50:00Z

While support for GWT is baked into Nuxeo, this native approach doesn't offer the flexibility I'm after to create composite applications where Nuxeo is just one source/sink of content. In previous applications, I have used a rest-client based approach using the RESTLET framework. However, I couldn't find any reasonable way to get the GWT datagrid control to play nicely using the restlet apporach. So, I switched gears, and returned to an HttpClient RPC approach for integrating GWT datagrids with Nuxeo.

What I do is bind the DataGrid to an AsyncDataProvider. The provider is set up in the corresponding presenter.

// Create a data provider.

dataProviderAsync = new LaborProfileAsyncDataProvider(SharedConstants.API_ACTION_LIST, eventBus);

dataProviderAsync.addDataDisplay(getView().getDataGrid());

addColumnSortAsyncHandling();

The provider's onRangeChanged method then contains the applicable RPC server call. Data is reflected into the datagrid control in the onSuccess method (updateRowData and updateRowCount). OnSuccess also fires an event that has a listener update on screen statistics.

// RPC Call

rpcService.getLaborProfileList(action, CurrentUser.getInstance().getUserName(),

CurrentSession.getInstance().getSessionId(),

CurrentUser.getInstance().getOwnerId(), searchFilter,

new AsyncCallback<ArrayList<LaborProfileRowFlat>>() {

@Override

public void onSuccess(ArrayList<LaborProfileRowFlat> result) {

if (result != null) {

filteredRowCount = result.size();

if ("".equals(searchFilter)) {

totalRowCount = filteredRowCount;

}

updateRowData(0, result);

updateRowCount(filteredRowCount, true);

eventBus.fireEvent(new UpdateLaborProfileStatsEvent());

}

});

The rpcService class prepares the command url (in url) and the Nuxeo transaction (in jsonString), and then handles the resulting JSON. The URL corresponds to a defined URL restlet pattern in Nuxeo (still using restlets on the Nuxeo side), from example the pattern <urlPattern>/laborprofile/{action}</urlPattern> is mapped to the Nuxeo restlet class that handles laborprofile transactions. The URL is the glue between the GWT caller and Nuxeo.

// MAKE THE CALL and PROCESS THE RESULT...

HttpEntity resEntity = HttpClientCall.execute(url, jsonString);

String resSource = "";

JsonNode rootNode = null;

JsonNode dataNode = null;

if (resEntity != null) {

...

The actual HttpClient call requires a bit more packaging (call to set-up security is removed) but is relatively trivial.

// MAKE THE CALL

DefaultHttpClient client = new DefaultHttpClient();

...

HttpPost postMethod = new HttpPost(url);

StringEntity reqEntity = null;

try {

reqEntity = new StringEntity(jsonString, "application/json", "UTF-8");

} catch (UnsupportedEncodingException e1) {

System.out.println("Error parsing jsonString");

}

if (reqEntity != null) {

reqEntity.setChunked(false);

postMethod.setEntity(reqEntity);

postMethod.setHeader("Connection", "close");

HttpResponse response = null;

try {

response = client.execute(postMethod);

} catch (ClientProtocolException e) {

e.printStackTrace();

} catch (IOException e) {

e.printStackTrace();

}

// PROCESS THE RESULT...

HttpEntity resEntity = response.getEntity();

return resEntity;

}

Straighforward enough once you have worked out all the details.

Wearable Computers and the MYO #getmyo @thalmic

2013-03-02T17:50:55Z

Wearable, guesture-based computing is a little off-topic from my normal blogging, but very cool and relevant (to me at least). Yesterday, Thalmic Labs, a Waterloo-based tech startup opened up pre-ordering for the MYO (getmyo.com). All the big tech blogs and publications covered the release and there are numerous articles about the MYO (Techcrunch, Wired, Financial Post).

The MYO represents a big step into the future - guesture-based control without the need for cameras. The applications of the MYO are limitless.

For those that think this isn't real - think again. The science behind the MYO is real, the product is real, and those backing the company are very real (and experienced). I suppose I may be a little biased given that my son Aaron is one of the co-founders, however, I believe this technology has the potential to take wearable computers to the mainstream.

Nuxeo Upgrade 5.3.2 to 5.6

2013-02-05T16:32:23Z

Once again I am impressed at the job Nuxeo has done to make the core migration process as painless as possible. Moving from 5.3.2 to 5.6 (through 5.4.1, 5.4.2, 5.5) is time-consuming but it produces a very predictable result. I think I read somewhere on the Nuxeo that this is the preferred approach, and it does make sense (at least to me). I create a bare-bones customization (schemas, doc types, UI, and a few other contributions) to enable me to load Nuxeo after each upgrade step and see that my data is preserved/available as expected.

Of course, that is not to say there isn't effort required in such a migration, especially when you've created extensive add-ons that rely on certain architecture/features/fields within Nuxeo. However, even that migration effort can be fairly easily scoped and understood.

My only real criticism of the process is migration documentation that explicitly spells out significant changes to the architecture or data model. Sorry, but trolling through java docs, changesets, or the like just doesn't cut it!

By way of example. From 5.4.1 to 5.4.2 the lock field (in the locks table) was removed and split into two separate fields (owner and created date). As part of the upgrade process Nuxeo is smart enough to split the existing field into the two new fields - unless of course it hits something unexepected and the process fails (so no lock information is preserved). Understanding what was happening and why took some time, which could have been avoided with a detailed change guide!

Time for NuxeoWorld North America

2012-10-29T14:13:53Z

A number of realizations got me to thinking that it's high time for a NuxeoWorld North America. Consider the following...

Recent Nuxeo press release about 40 percent growth in business in North America
Nuxeo CEO resides in the US
There are three offices in the US (and only one in Europe)
For what is effectively a one-day conference (not including the Dev day) the time and monetary costs are too high for many outside Europe to attend

If Nuxeo is serious about growth, serious about feeding the community, serious about listening to customers, and wants to effectively compete with Alfresco and others then I think it's time to add NuxeoWorld part deux. Other vendors do exactly the same thing because they realize the importance of bringing clients, developments and vendors together face-to-face. Conferences help create and strengthen relationships, provide a forum for direct dissemination of important information, as well as gathering first hand real-world feedback.

I know there is a certain critical mass required before such an event can be considered, but certainly that point must be here (or at least very close).

Nuxeo Studio as a separately priced SAAS?

2012-10-26T11:49:01Z

It's been a while since I have had any breathing room. I wish I had the cycles to attend NuxeoWorld in Paris, but I simply don't. However, when NuxeoWorld comes around it does get me to thinking about things Nuxeo could do better.

While I do most of my Nuxeo development in the IDE there are some tasks that are simply far more efficient in Studio (e.g., basic schema and document prototyping). The tool has definite value. Problem is, there isn't a separate offering just to use Studio - you have to buy a full support package (yes, there is Connect Base but that still has Nuxeo Maintenance wrapped in). As a developer and integrator this just doesn't make sense since I do not need or require technical support (that is included in the support package).

I would like to see Nuxeo sell Studio as a separately priced Software as a Service (SAAS) offering, much like Jira and many others. I think this approach would increase adoption and drive up customer support revenues.

I suspect many Nuxeo developers would be willing to pay a monthly usage fee -- especially if they have bought into the SAAS model.

Maybe this will be announced at NuxeoWorld, although that's probably just my hopeful thinking.

Great GWTP resources...

2012-08-20T21:35:17Z

I can't remember how I came upon GWTP, but I am sure glad I did. GWTP provides an additional level of structure, abstraction, and code generation that makes it much more enjoyable to build GWT interfaces and (more importantly) connect to back-end service/data providers.

But like any framework there is a learning curve. I learn best by example, so over the last months, I have been searching for practicals tutorials and examples. The two best resources I have found to date:

Dani's YouTube video series on GWTP http://www.youtube.com/watch?v=Gm-RO-cmsEQ
Uptick tutorial and working application http://uptick.com.au/content/gwt-login-security

There are many more great resources for GWTP (including the project site), but I found these the best at getting started and then diving deep into the inner workings of a real application. Many thanks to the authors of GWTP, Dani, and the folks at Uptick!

Nuxeo 5.6 - One Time Event Trigger example

2012-08-08T22:40:52Z

Four parts to creating and using a one-time event (leveraging the Nuxeo SchedulerService):

1. Register a 'scheduled' event - in this case the event will fire in 10 minutes time. The event that fires is called myNuxeoEvent in this sample.

    public class FireInTenMinutes {

   public void task() {

        SchedulerService service;
       try {
           service = Framework.getService(SchedulerService.class);
       } catch (Exception e) {
           e.printStackTrace();
           return;
       }

        ScheduleImpl schedule = new ScheduleImpl();
        schedule.cronExpression = "0 0/10 * * * ?";
        schedule.id = "test1";
        schedule.username = "Administrator";
        schedule.eventId = "myNuxeoEvent";
        schedule.eventCategory = "default";
        service.registerSchedule(schedule);

   }

}

2. Create a listener to respond the event created above. In this example all I am doing is wirting to the log when the event fires. Since we want this to be a one-time event, I unregister it (again using the SchedulerService) once the event is handled.

    public void handleEvent(Event event) throws ClientException {

       if (!("myNuxeoEvent".equals(event.getName()))) {
           return;
       }

       log.info("*** Fired in 10 minutes! ***");

       // remove the one-time event after it fires
        SchedulerService service;
       try {
           service = Framework.getService(SchedulerService.class);
       } catch (Exception e) {
           e.printStackTrace();
           return;
       }

        service.unregisterSchedule("test1");
   }

3. I have to register the listener in a contribution file.

    <extension target="org.nuxeo.ecm.core.event.EventServiceComponent" point="listener">

      <listener name="test1" async="false" postCommit="false"
           class="com.concena.mckellar.coreapp.delete.TenMinuteListener" priority="135">
           <event>myNuxeoEvent</event>
       </listener>

etc...

4. And finally, you'll need to initiate the event from your existing code. In my example I add two lines to an existing event handler.

    protected void handleTrashCompactorEvent(Event event) {
        try {
           RepositoryManager repoMgr = Framework.getService(RepositoryManager.class);
           Repository repo = repoMgr.getDefaultRepository();
           RemoveDocuments removeDocs = new RemoveDocuments(repo.getName());
           removeDocs.runUnrestricted();
       } catch (Exception e) {
           log.error("*** Error removing documents in trash *** error: " + e.getMessage());
       }

       // fire test event in 10 minutes
       FireInTenMinutes fireInTen = new FireInTenMinutes();
       fireInTen.task();

That's all there is to it!

Nuxeo Event Trigger - firing a simple one-time event

2012-08-08T13:39:56Z

I see that in Nuxeo 5.6 the SchedulerService has moved packages and undertaken some renaming. I disovered this after looking further into a question on Nuxeo Answers.

In order to use the Scheduler Service effectively it would be extremely helpful if the underlying SchedulerServiceImpl registerSchedule method could handle the case where a CRON expression (in the schedule object) is null. When the CRON expression is null then a SimpleTrigger could be created that could be run 1 .. n times.

Something like...

if(cronexpression==null)
use SimpleTrigger
else
use CronTrigger
}

This would nicely support one-time events created in code without having to use a CRON expression. Would be even nicer if there was a way to automatically remove the one-time scheduled service on firing to avoid having to manually unregister. Maybe there's an easier way?

First Impressions - Nuxeo 5.6

2012-06-27T11:46:01Z

Last week I downloaded a Nuxeo 5.6 build. After downloading, it took me only a few minutes to get the instance up and running - pure vanilla with the exception of using Postgresql 9.1 as the database. I have only had a bit of time to play with the release and explore some of the changes but I have to say that my first impressions are very favourable.

Running Nuxeo 5.6 on the same server as I run 5.5 using the same components (DM, Collab, and DAM) the 5.6 server started slightly faster (this was not a scientific comparison but the results seem consistent); the start time on the server went from 21 sec (5.5) to about 19 sec (5.6); server spec - see below
I felt the responsiveness (performance of the UI) was improved significantly (again this is my out of the box impression -- I may be crazy). I did find that the default navigation tree jumped a little (like a pixel shift) and that was a bit disconcerting. However, remember I am not looking at released code and so there is probably much last minute testing and tweaking underway in the Nuxeo development bunker
The UI takes on a much more polished look than its predecessors. This is reflected in updated graphics and in Action menus that just look more professional (it's too bad this new look doesn't seem to extend to sub-actions - at least out of the box - although I haven't dug under the covers and maybe this is easily configured)
Reorganization of the default DM UI makes for (in my opinion) better use of available space - some of that's achieved with icons and some is achieved through a combination of better contrast and better presentation of simple data (like current document state and contributors)
Every time I visit the Admin Center there is cool (and more importantly useful) functionality exposed - I only sped breifly through this section - enough time to see a number of new tabs, functions and the same UI improvements as DM
I didn't, however, see any outward facing improvements in the DAM interface (anybody help here - am I missing something?)
The cookie trail seems to have shrunk in size, which personally I like, because I always like to see the entire path up there, but it may be too small for some
One part of the UI I would really like to see streamlined is security administration. This is one part of the UI that has been with the product for as long as I have been using it, and in my opinion it's just clunky - needs to be more intuitive and easier to manage. Not simple I know, but given the other UI improvements hopefully somebody will turn their attentions to this area of the UI in an upcoming release
Really like the new icons although it will take a bit to remember which one is which (this isn't a complaint, rather an observation)
Just haven't had time to make it through Social Collaboration in any detail but the new dashboard looks great and although it may seem trivial, I really like the idea of a built-in voting system
I know there are numerous internal improvements (some obviously reflected in ui and load speed) to look forward to, but not this time around

The pre-release code seems extremely stable - out of the box, running in less than 15 minutes, and no crashes! Spending a few hours looking at 5.6 scratched the surface but was well worth the investment in time.

Server Spec: i7, 8 cores, 32GB RAM, SSD Boot/App disk, 7200RPM SATA for data, running CentOS 6.2 64 bit.

Nuxeo Roadmap

2012-06-07T19:57:58Z

I had signed up to attend, but ended up missing, the Nuxeo roadmap session. But my customers come first and I had to take of a few time-sensitive issues. C'est la vie. The session is up on the site so at least I could watch after-the-fact. Apart from what I have previously written about with respect to DAM, I had some more general questions about the future of Nuxeo that I would have liked to ask. At least one was answered during the webinar and another touched on briefly.

There are many third party libraries that are part of Nuxeo. Some of these libraries are fairly old (in technology terms) and I think they need to be updated much more frequently. Libaries for PDF, image manipulation, metadata extraction, and more need regular updates. I would like to know when/if these will become part of the regular hotifx / release cycle.
Integration of Studio and IDE with full two-way engineering. The ability to move back and forth between the environments would have great value. It looks as though 5.6 will start down the path but I will have to wait and see whether there is value in the initial offering.
Whether on premise or in the cloud, performance is of critical concern - especially with huge digital assets. I would like to know specifics on plans to improve overall imaging and asset management performance.
Nothing specific in mind, but I would like to see fewer error pages popping up in the application. And when they do it would be nice to have a way to return to the context prior to the application cacking. Just wondering if there is any work under way to tighten up and improve error handling.
More into the future - any thoughts or plans to support any NoSQL database variants?

I was glad to see more work in the mobile space as this is critical in my opinion to delivering focused, effective content applications to users regardless of where they are!

Hopefully I get a chance to attend the next roadmap session.

Beyond Nuxeo 5.6

2012-05-28T18:00:19Z

Looks like the 5.6 release of Nuxeo has some promising ui/usability improvements. No doubt there will be improvements in other areas as well. Hopefully I will have some time in the not too distant future to investigate.

I'm looking beyond 5.6, specifically with my new DAM goggles on. There is demand for industrial strength DAM across most verticals. In some verticals, digital asset management underpins the business itself, and in these situations industrial strength DAM is critical. I would like to see a stronger focus on improving the DAM product (and underlying CAP/imaging components).

I think it's time for Nuxeo to step up efforts to improve DAM over and under the covers - to get it on a level playing field with commercially available DAM solutions.

What do i think are the most important areas on which to focus?

1. Format agnosticism.

Whether it's a raw image from a Nikon camera, a multi-layer PSD, or one of a myriad of other formats the DAM solution needs to be able to handle it,

2. Size agnosticism.

Industrial strength DAM products must handle 1.5GB Tiff files just as eloquently as they handle a 5MB Jpeg. While processors, RAM, and disk speed will determine overall performance, the solution should have workarounds for any reasonably sized hardware or virtual machine. This also means finding and eradicating all software-based limitations for handling files > 2GB in size.

3. Metadata - all of it.

Filtering and finding images is one of the most important functions (in my opinion) of a digital asset system. Using image metadata for searching and filtering requires efficient automated extraction. And, it means that every type of common metadata must be easily extracted from supported image types. This includes, but is not limited to, XMP, IPTC, and EXIF.

4. Performance Profiling data and Sizing Charts

The challenges with moving and processing multi-GB files are much different than those of smaller files (where transaction times - even aggregrated - or not all that significant). It would be extremely useful if Nuxeo published recommended configurations for all Nuxeo applications, but DAM specifically. The recommendations could include JVM sizing, Database configuration, OS configuration, Imaging component tweaks, RAM, disk usage, etc. This is a big complex area, but some base level of recommendations would be helpful as a starting point.

5. Speed, Speed, Speed

With big images, and lots of them, processing speed is critical. I have seen a number of opportunities in the core Nuxeo imaging code to improve overall transaction speed. One of the biggest areas for possible improvement is the imaging library - especially when spinning off multiple Jpeg versions of the original image. Lots of opportunity here.

6. Third party Library Updates - More Frequent/Alternates/Plugins

Libraries that support third party image processing (ImageMagick, metadata-extractor, etc.) have to be kept up-to-date, preferably as part of point releases. This ensures that newer image formats can be handled with minimal fuss. Maybe it's time to look at alternatives to some of the image processing components. Is Google's OiiO a possible alternative to ImageMagick? Is there a way to abstract the third party image library in such a way that OSX users could take advantage of the CoreImage library for image processing (which is far faster than ImageMagick equivlents)?

[Added May 28 2012:

7. Parameter driven functionality

An example... When an asset is ingested into the DAM repository three images are generated - an original size JPEG, a Medium size JPEG, and a thumbnail. The number of images generated and their sizes are hard-coded in the application. It would be nice if the configuration of this functionality were exposed in a configuration document, allowing additional renditions, lower resolution renditions, different formats, etc. In my opinion this would greatly improve the flexiblity and applicability of DAM.]

What else can Nuxeo improve in the current DAM product to make it more industrial strength?

Cheers,

Bruce.

Nuxeo, DAM, ImageMagick and really big files

2012-04-29T20:57:54Z

Dam frustrating is a good way to describe my last three days. I have been working to tune the Nuxeo DAM import for very large images … frustrating because there are many pieces, moving big images takes time, and debugging/document the whole process introduces a natural latency.

Nuxeo import relies on ImageMagick for image-related tasks (e.g. resizing and cropping) - during import ImageMagick is used heavily and is by far the most time-consuming element of the import process. If ImageMagick slows then so does your import.

Nuxeo DAM, for that matter the core imaging code, works really well with most images. "Works really well" extends to the integration with ImageMagick. But take a step into the world of very large/complex image files and the works well fantasy world takes a halting step into reality.

What do I mean by very large image (or complex) files? Three possible interpretations:

In absolute terms, any image file over 2GB is very large (well, technically anything over 2^31-1, which is the limit of an int value in Java). This limit is important because any code that uses int values for file size or related processing will be limited to 2GB files. There are still some limitations within Nuxeo when it comes to uploading very large files (not just images) - e.g., in DAM I can import a 3.5GB video file through the importer but the same import fails when attempted through the UI. I do know that Nuxeo developers have been working actively to remove these barriers.

In relative terms, very large can be any image that is big enough (or the requested operation complex enough) to require more physical memory during image processing than your OS can make available. For example, images in the 1GB range can be a problem on a Windows box with 4GB of RAM (assuming Nuxeo server is also running). ImageMagick runs fastest when the entire pixel map can be loaded into RAM. If the entire pixel map can't be loaded into memory then image processing slows dramatically – that’s because ImageMagick will page parts of the image in/out of memory to complete the requested task.

Complexity of the image file itself can exacerbate the situation. For example, multi-layered PSD images require far more processing time than a single layer file of the same size. To the ImageMagick identify command each layer is effectively an image of its own, with its own metadata. So it takes ImageMagick much longer to troll through multiple images to extract the required information.

Add to this complexity the differences in memory allocation/reservation of different operating systems and it makes it challenging to tune Nuxeo DAM and ImageMagick.

I have spent the last three days running through numerous configurations, file sizes, limits, memory allocations, etc. trying to better understand the issues in order to move forward. What I found is...

So long as you're running VCS then Postgres tuning is not really important for image processing (of course it is very important for other reasons)
Technically, the amount of RAM you set aside for Nuxeo will not limit the size of images you can import, but practically speaking, if you don't allocate enough RAM to reflect the size and volume of images imported, displayed, etc. in your system then your DAM solution will be less than useful. The RAM required is dependent on numerous other factors so it's difficult to be entirely prescriptive
The amount of physical RAM available to ImageMagick is absolutely critical to timely image processing. Not enough RAM and ImageMagick will page to disk, run times will increase dramatically and it's likely your Nuxeo transaction will timeout. The end result … nothing will be imported (the transaction will be rolled back). You might want to set upper limits on the resources available to ImageMagick (using environment variable) so all available RAM isn't gobbled up
The speed of your disk is also important. Slow disk = slower image processing. Longer to read, longer to write, longer to create any interim temp files. If your disk is on the network then make sure it's on a high speed connection. You might want to consider a solid state drive for ImageMagick temporary files.
CPU speed, number of cores, and number of threads allocated (for ImageMagick) must also be considered - especially from a global perspective, since multiple image commands can be running simultaneously.
The location of ImageMagick temp files is important. ImageMagick commands produce temp files, the location of which can be controlled with the MAGICK_TEMPORARY_PATH environment variable. Set the path variable to a disk with lots of free space, preferably not the same disk as your OS and Nuxeo are on.

Each case is difference, but the end result for Nuxeo-ImageMagick import tuning: expect to spend extra time understanding your requirements, get ready to buy more memory, and be prepared to put in some effort to get everything running quickly!

Nuxeo Community Survey - Any Interest?

2012-03-30T14:18:09Z

I have been kicking about the idea of running a community driven survey on Nuxeo (the product suite and the company). Is anyone else interested in understanding how Nuxeo is being used, which versions are being used, what software has been integrated with/to Nuxeo, what people like/don't like about Nuxeo, suggestions to improve, etc.

My thinking is as follows:

1. Gather some basic demographics (products, versions, number of users, geography, etc.)

2. Gather feedback on how Nuxeo is going as a company (quality, responsiveness, etc.).

3. Gather feedback on individual products (likes, dislikes, ideas to improve, etc.)

4. Gather information on how products are being used (purpose, integrations, etc.)

5. And free format feedback.

Once collected I will generate summary report. The report will be shared with Nuxeo as formal community feedback, and a copy of the report will be emailed to all who participated in the survey. I will also post the survey on my site for those who don't provide contact information. Published results will be completely anonymous.

This will take some effort on my part but I think it will help the community, those considering Nuxeo, and it can help guide the future of the product?

Am I crazy (I probably am) or would this have some value? Also, let me know if you have suggestions for specific questions or areas of interest to include in the survey.

Thanks.

Nuxeo Hotfixes - An Idea

2012-03-22T19:17:17Z

Today on Nuxeo answers a user asked about the availability of the latest hotfix package for 5.5 (link to Nuxeo answers question). The reply, as I expected, was that packaged hotfixes are for paying customers only. Absolutely understandable, for although Nuxeo is open source, Nuxeo the company is a business and needs to make money to stay in business and flourish. Hopefully everyone who has invested time and energy in Nuxeo can appreciate this.

But this got me to thinking, maybe there is an additional subscription model for small businesses and individuals, where there is no additional support provided, but hotfixes are sold as-is for a fixed price (via credit card or paypal). To Nuxeo this becomes a transaction requiring no additional labor or cost. Incremental revenue, happier users, and a chance at growing the customer base when users reach a certain size where they feel a more formal support arrangement is warranted.

In my mind, if the cost of a hotfix were set reasonably it would far outbalance the cost of having to build from source and then deploy on one or more servers (and having to deal with any resulting issues). Would $150 per hot fix be reasonable?

I can see the downside to this of course - create more hotfixes to generate more revenue... that would be evil :-)

Ok, so what about an annual hotfix only subscription. Look at the last few releases and the number of hotfixes per year on average and let's say $1500 per year for hotfixes only - but no technical support (other than that provided in forums and answers).

Thoughts?

Cheers, Bruce.

Leveraging ImageMagick with Nuxeo

2012-03-14T05:06:46Z

Until recently I never really appreciated the value and power of ImageMagick. This is one slick piece of software (or collection of image transformation tools).

The base integration with Nuxeo is good, but there is still plenty of opportunity to improve. The need for speed (and flexibility) becomes very evident when dealing with 250MB images, TIF images, IPTC/EXIF/XMP metadata, and with images that have a CMYK color space.

Quick Fix...

ImageMagick is integrated with Nuxeo, or more accurately loosely bound to, via a series of command line calls that perform the image-related magic. Nuxeo expects to find the ImageMagick tools on the system path, while the loose binding comes by way of a contribution that associates Nuxeo command names with ImageMagick commands and parameters. Part of the default Nuxeo contribution is shown below…

<?xml version="1.0"?>
<component name="org.nuxeo.ecm.platform.picture.commandline.imagemagick">
    <require>org.nuxeo.ecm.platform.commandline.executor.service.defaultContrib</require>
    <extension target="org.nuxeo.ecm.platform.commandline.executor.service.CommandLineExecutorComponent”
                         point="command">
        <command name="identify" enabled="true">
            <commandLine>identify</commandLine>
            <parameterString> -ping -format '%m %w %h %z' #{inputFilePath}</parameterString>
            <winParameterString> -ping -format "%m %w %h %z" #{inputFilePath}</winParameterString>
            <installationDirective>You need to install ImageMagic.</installationDirective>
        </command>

       <command name="resizer" enabled="true">
           <commandLine>convert</commandLine>
          <parameterString>-flatten -resize #{targetWidth}x#{targetHeight} -depth #{targetDepth} #{inputFilePath}[0]
                          #{outputFilePath}</parameterString>
          <installationDirective>You need to install ImageMagic.</installationDirective>
       </command>
Etc.

I included this chunk of code because it demonstrates an issue with ImageMagick and the flexibility of Nuxeo (and ImageMagick) to address the issue.

In the example, the Nuxeo “resizer” command gets tied to the ImageMagick “convert” command. If the input picture is a TIF file then resizer will convert the image to a JPG and resize as required. This causes a problem when the TIF image has layers/masks. Unfortunately, converting from TIF to JPEG and resizing at the same time results in the loss of layers/masks in the resulting JPEG image – not so good.

There is a workaround, and that involves splitting the resizer command into two parts. Luckily both Nuxeo and ImageMagick are fine with using pipe-lined commands. So, what I split the single default resizer command into a conversion and then a re-size. I overrode the existing contribution with the following….

<command name="resizer" enabled="true">
     <commandLine>convert</commandLine>
     <parameterString> #{inputFilePath}[0] jpg:- | convert - -resize #{targetWidth}x#{targetHeight} –depth
                                    #{targetDepth} #{outputFilePath}</parameterString>
     <installationDirective>You need to install ImageMagick</installationDirective>
</command>

In the new “resizer” the input file first gets converted to a JPEG file (represented by the -), which is then piped into a convert command to resize the JPEG as required. This may cause slightly more work, but TIF files now get converted and resized as expected.

Resolving the TIF conversion issue with a simple contribution is a fairly simple solution.

One or the Other...

Unfortunately, not all fixes are so straight forward. The “CropAndResize” command is used by the tiling service to facilitate annotations. Crop and resize issues a pipelined ImageMagick command (stream | convert … see below) that takes an input stream and converts the image to a specified size using a given color map. Problem is – the color map is hardcoded as RGB – which is an issue for any image in a CMYK color space.

<command name="cropAndResize" enabled="true">
      <commandLine>stream</commandLine>
      <parameterString> -map rgb -storage-type char -extract #{tileWidth}x#{tileHeight}+#{offsetX}+#{offsetY}
                                    #{inputFilePath} - | convert -depth 8 -size #{tileWidth}x#{tileHeight} -resize
#{targetWidth}x#{targetHeight}! rgb:- #{outputFilePath} </parameterString>
      <installationDirective>You need to install ImageMagic.</installationDirective>
<command/>

The –map rgb in the cropAndResize command can be changed to –map cmyk (and the piped output changed to cmyk:-) … and CMYK images will now appear properly for annotations. However, RGB images are no longer tiled correctly. In this case programmatic changes are required to determine the color space of the image being processed and then to pass the appropriate color map value to cropAndResize as a parameter. Getting the color space could be part of an existing Identify command (where all image objects are picked up) or it could be done in a separate process using “identify –format %[colorspace] filename.tif” (although this will incur extra processing costs).

To make this work the image tiling package and imaging core package need to be modified.

Big Image Efficiencies... next time

Finally, when it comes to dealing with big files in Nuxeo (100MB+) there is room for improvement. I haven’t yet walked through the whole process but I have watched the number of temp files that get created during the ingestion process. At first blush it appears that there is an opportunity to make this more efficient – which is really important for large files! However, I will have to dig into this another time.

Assessing the Cost of Nuxeo versus Commercial Alternatives

2012-03-07T18:36:28Z

When customers ask about the cost of Nuxeo I always answer in terms of value. The primary reason for this is simple; I think that the value of an open source approach is by far the more important discussion and will underpin any discussion of cost (or investment). However, understanding cost is critical to making any investment!

The following table outlines the costs associated with Nuxeo versus traditional commercial solutions. Arguably there are other categories which could be included the list. However, these are, I think, the items with the most cost impact.

	Nuxeo	Commercial	Comments
License Fees	None	Server and/or user license fees. Adding more users or servers dramatically increases the costs.	Open Source – no initial fees, but more importantly no barriers to implementing more servers or adding additional users (inside or outside your company)
Maintenance Fees	None	Typically 20-25% of license fees. More users and servers increases annual maintenance costs as well	For commercial solutions maintenance fees are required just to get access to new releases of software.
Connect Subscription	Convenience features and access to online Studio development suite	None	The connect subscription is optional, but is required if annual support is required
Annual Support	Per Application irrespective of number of servers or users. Multiple levels of support are available with requisite Service Level commitments.	None in standard maintenance model	Nuxeo annual support is optional. In practice, commercial software often has additional premium support costs as well as those not covered by maintenance.
Development	Required to implement customer’s business requirements. The size and scope of this effort is tied directly to requirements. Simple customizations with Studio, more complex with Nuxeo IDE.	For true off the shelf software there may be no development required (or possible!).	Nuxeo-based solutions are built to suit your business requirements, rather than having to mold your requirements to a commercial solution. It’s typical for some level of integration development (at a minimum) with commercial solutions.
Configuration	Included in development	A separate step in commercial products to configure fields, content types, view, etc.	Configuration allows for some customization of Commercial solutions, but scope and flexibility are limited.
Initial Implementation	Tied to scope	Tied to scope	Assumed neutral in terms of cost, however, this is not always the case.
Server Updates (hot fixes)	With annual support the application of server hot fixes is (almost) trivial. Otherwise, manual updates are still possible, but require additional effort.	Included with annual maintenance, but the effort to install varies based on the vendor and the product.
Upgrades	There will be an ongoing cost to maintaining custom developed code.	There will be a cost to maintaining any custom-developed integrations.	Difficult to compare in the abstract, however, Nuxeo is architected in a way that promotes extension and redefinition of function while maintaining ease of upgrade.

The bottom line: a Nuxeo-based solution will likely yield a cost savings over a commercial alternative. In the worst case scenario the cost difference may be neutral. However, as I mentioned at the start, I think that the real benefit (value) of Nuxeo comes from its flexibility, extensibility, scalability, performance, standardization, build-to-purpose, rapid pace of development. While estimating cost is necessary, defining the value of Nuxeo to your organization is imperative!

I'm very interested to hear how others feel about this...

Extracting EXIF, IPTC, XMP and more with Drew Noakes metadata-extractor-2.5.0-RC3

2012-03-05T21:28:01Z

Nuxeo already uses an earlier version of the metadata-extractor library. Problem is, this version is somewhat dated (circa 2006/2007) and has very limited support for the broad range of embedded metadata. Well, the new 2.5.0-RC3 version looks to address this limitation by adding support for a much broader array of embedded metadata standards - including XMP! Here's a link to the metadata-extractor project. Thanks Drew!

There are enough structural changes between 2.3.0 and 2.5.0-RC3 that simply replacing the JAR will not work. In the case of Nuxeo that would have meant finding and fixing all the touch points with the metadata-extractor. I opted for a simpler option (that in no way impacts existing code).

1. Downloaded the sources to metadata-extractor-2.5.0-RC3 and the changed the package name from com.noakes.* to com.noakes.custom.* . I then recompiled to create a -RC3.1 jar.

2. Copied the -RC3.1 jar into my nxserver/lib directory (along with xmpcore.jar - comes with RC3 jar download)

3. Overrode and then modified PictureChangedListener to explicitly use -RC3.1. Then added the following code (snippet only). All of this code runs after Nuxeo has processed the blob and set what metadata it can. Obviously this can be made smarter to check to see if (as is the case for JPEGs) Nuxeo found and set the EXIF/IPTC metadata. And of course, longer term it will be best if the newer 2.5.0-RC3 is baked into Nuxeo!

                ImagingService service;
                Map<String, Object> map;
                try {
                    service = Framework.getService(ImagingService.class);
                    BufferedInputStream in = new BufferedInputStream(blob.getStream());
                    Metadata metadata = ImageMetadataReader.readMetadata(in, true);
                    exifDirectory1 = metadata.getDirectory(ExifIFD0Directory.class);
                    // metadata.getDirectory(ExifInteropDirectory.class);
                    exifDirectory3 = metadata.getDirectory(ExifSubIFDDirectory.class);
                    iptcDirectory = metadata.getDirectory(IptcDirectory.class);
                    // metadata.getDirectory(XmpDirectory.class);

                    if (exifDirectory1 != null) {
                        Date dateTimeOriginal = getExif1Date(exifDirectory1.TAG_DATETIME);
                        int width = getExif1Int(exifDirectory1.TAG_X_RESOLUTION);
                    :
                    :

The getExif1xxx functions do the necessary checking to make sure that tag exists and conversion to proper type. For example...

      private int getExif1Int(int tag) {
                  int result;

                  if (exifDirectory1.containsTag(tag)) {
                       try {
                          result = exifDirectory1.getInt(tag);
                       } catch (MetadataException e) {
                          result = 0;
                       }
                   } else {
                       result = 0;
                 }

               return result;
               }