XRootD Collaboration Meeting held at CERN 29-October-2013

Attendees:

Discussion Topics

Release 3.3.4

No RPM compatibility issues have been found. It provides a smooth installation transition.

Release 3.3.5

We are planning another patch release. Minimally, this will contain:

SL6 Memory Allocation Issues

Due to the change in glibc malloc in SL6, heavily threaded programs may see a dramatic increase in virtual memory use (not real memory). This can cause batch job cancellation on many sites and causes severe problems for PROOF jobs on nodes with limited swap space.

The immediate solution is to set the special envar MALLOC_ARENA_MAX to a value of 1 for client jobs and 4 for the servers. This must be set before the program starts. This is not particularly practical in many cases and requires a consultation every time someone experiences this problem.

The long term solution is to switch to a better memory allocator. Two are available: tcmalloc (Google) and jemalloc (Facebook). Of the two, it appears the jemalloc is superior. Unfortunately, neither of these are natively installed as part of the Linux distribution and require a separate RPM install. Furthermore, it is difficult, to link the special malloc library ahead of glibc and requires that it be set in the LD_PRELOAD envar prior to running a program.

Fortunately, Atlas makes tcmalloc the default for Athena jobs and CMS makes jemalloc the default for CMSW jobs. That leaves a swath of others exposed.

The action items we are undertaking to address this issue:

Release 4.0.0

We are tentatively targeting a release candidate no later than the middle of January 2014. Prior to that time we are considering making a preview RPM available via the xrootd.org site. The preview RPM may have features added to it prior to actual release candidate availability.

This is a major change in the XRootD package and is not binary compatible with previous releases. This isn’t a problem client-side but plug-in writers will need to recompile server-side.

Because of backward compatibility issues we are constrained how we release the package via EPEL. Currently, our plan is to release the package as xrootd4 (as opposed to xrootd) and only rename the package back to its original name in EPEL 7 which, in itself, is a non-backward compatible release. Some major changes people will see (though, in fact, transparent) are:

Do You Need To Upgrade?

That was the question posed by Doug Benjamin. In fact, he pointed out that it is very difficult for an administrator to decide. Our solution to this issue is to place a link next to each release on xrootd.org labeled something like “Upgrade?” which will point to a page describing the changes that may be of interest in various categories (e.g. performance, federation, etc.). Andy will be responsible for creating the page after Lukasz sends him the release notes for a particular release. So, there may be a couple of days delay between the release being posted and the upgrade page being available.

Miscellaneous Action Items