/[pdpsoft]/trunk/userspace-package-management/notes.org
ViewVC logotype

Annotation of /trunk/userspace-package-management/notes.org

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1904 - (hide annotations) (download)
Tue Aug 31 12:32:09 2010 UTC (11 years, 8 months ago) by dennisvd
File size: 2542 byte(s)
initial checkin of scripts

1 dennisvd 1904 * User space package management
2    
3     The PoC (Proof of Concept / Packages of Choice) has always been a centrally provided collection of
4     packages for end users. The alternative--user space package management-- allows the VO software
5     managers to handle VO specific software installations per site, thus gaining control over versions
6     and build parameters.
7    
8     The pkgsrc system, by NetBSD, is being tried as the basic packaging toolset for managing updates,
9     dependencies, etc.
10    
11     Not all science software is readily available in pkgsrc, but with some work any regular package
12     that follows the usual configure/make/make install mantra can be incorporated.
13    
14     Extra tooling around pkgsrc is needed to interact with Grid systems, because software management
15     is done by sending grid jobs with a software manager proxy (to gain the appropriate user mapping),
16     and all packaging/updating/etc work needs to be done inside the grid job.
17    
18     ** Grid jobs
19    
20     The general layout of the grid jobs is as follows:
21    
22     - test if we have the right permissions in the VO_SW_AREA (and set umask 002)
23    
24     - test for the presence of pkgsrc
25     - if not present: fetch and install
26     - if present: upgrade if necessary
27    
28     - run the vulnerability admin tool
29    
30     - see to the actual request (install, remove, etc.)
31    
32    
33     ** management and reporting
34    
35     As a VO software manager you are interested to keep track of all
36     the installed versions of packages on all the available sites. So
37     one level up from the pkgsrc-cmd.sh scripts run in grid jobs is a
38     set of tools to automate the submission of jobs to a known
39     collection of resources, and generating aggregated reports.
40    
41     On top of this we could have a service that periodically runs these
42     tools and maintains and serves the generated reports.
43    
44    
45     ** Design of get-site-status.sh
46    
47     This script will probe sites to find out the status of pkgsrc installations.
48     It will launch pkgsrc-cmd jobs, cycle to wait on the results, and collect
49     the output in a human-readable form.
50    
51     Some design principles:
52    
53     - the script is meant to be run as a batch job, so it can be used in a crontab file.
54     - the output can be text, html, or otherwise
55     - resources are listed in a data file,
56     - credentials to run the job under are presumed to be available. The user is
57     responsible for keeping credentials up-to-date.
58     - only one job per site should be run at a time; this goes especially for the jobs
59     that install packages as simultaneous writes may result in corruption of the database.

grid.support@nikhef.nl
ViewVC Help
Powered by ViewVC 1.1.28