/[pdpsoft]/trunk/nl.nikhef.pdp.lrmsutils/doc/lcg-info-dynamic-scheduler.txt
ViewVC logotype

Contents of /trunk/nl.nikhef.pdp.lrmsutils/doc/lcg-info-dynamic-scheduler.txt

Parent Directory Parent Directory | Revision Log Revision Log


Revision 108 - (show annotations) (download)
Mon Mar 2 14:24:02 2009 UTC (13 years, 5 months ago) by templon
File MIME type: text/plain
File size: 6984 byte(s)
fix for savannah bug 38195 : old FQAN format replaced by new one.

1 FOR THE IMPATIENT
2
3 Just installing the RPM should result in a working system.
4
5 You may want to look at the "vomap" section below if your
6 unix group names are not the same as your VO names.
7
8 OK THE REAL DOCS
9
10 This is an implementation of per-VO estimated response times and free
11 slots for the LCG 'generic information provider' framework.
12
13 HOW IT IS SUPPOSED TO WORK
14
15 There are two parts to the system.
16
17 One part (lcg-dynamic-info-scheduler program) contains the algorithm
18 that computes the response times. This part doesn't know the details
19 of the underlying batch system, so that the estimated times are as
20 independent as possible from the various LRMS.
21
22 The second part is the LRMS specific part. This gathers information
23 from the LRMS and writes it out in a LRMS-independent format. There
24 are two of these critters; one for the LRMS state, and one for the
25 scheduling policy.
26
27 A typical deployment will consist of the generic part, and two LRMS-
28 and/or site-specific plugins for this second part. These plugins need
29 to write their information in a specific format that is the same for
30 all LRMSes/schedulers. For the LRMS state information, see the file
31 lrmsinfo-generic.txt; for the free slot information, see the file
32 vomaxjobs-generic.txt.
33
34 WORKING EXAMPLES
35
36 See the doc directory, where a working set of files is included.
37 YAIM sets these up for your site by default, if you use YAIM.
38
39 DEPLOYMENT
40
41 The RPM building takes care of making sure that the scripts can find
42 the support modules (e.g. lrms.py, EstTT.py). If you decide to move
43 these modules somewhere else (relocate the RPM) then you will need to
44 adjust the paths in the script.
45
46 Aside from that the dynamic-scheduler plugin needs a configuration
47 file, which is described below. Finally, one needs to drop the ERT plugin
48 (lcg-info-dynamic-scheduler-wrapper) into the GIP plugin directory,
49 typically
50
51 /opt/lcg/var/gip/plugin
52
53 CONFIG FILE
54
55 The format of the config file is like a Windows INI file; for the
56 exact format, see the documentation for the ConfigParser module in the
57 Python standard library. Concisely it looks like this:
58
59 [Main]
60 static_ldif_file : value # required
61 vomap : # required
62 unixgroup1 : vo1
63 unixgroup2 : vo2
64 [ ... ]
65
66 [LRMS]
67 lrms_backend_cmd : value # required
68
69 [Scheduler]
70 vo_max_jobs_cmd : value # optional
71 cycle_time : value # optional
72
73 Here is an explanation of each of the options.
74
75 [Main] static_ldif_file
76
77 Set to the full pathname of the static LDIF file used by the GIP
78 system. It is used to find out which CEs are in the system, and which
79 VOs are supported by which CE. You can find this in a normal
80 deployment under
81
82 /opt/lcg/var/gip/ldif/static-file-CE.ldif
83
84 [Main] vomap
85
86 On many systems, control of access to the LRMS queues is done by unix
87 group names, associating a single unix group to a set of pool accounts
88 belonging to a particular VO. Sometimes the unix group names are not
89 the same as the VO names, "VO name" here being the thing that appears
90 in the GlueCEAccessControlBaseRule lines. The vomap is how
91 information coming from the LRMS backends is modified so that the ERT
92 and free-slot computations can be done using VO names found in the
93 static LDIF file, rather than having to make site-specific
94 customizations to either the LRMS backend plugins or to the ERT
95 scripts.
96
97 Specifically, if all relevant unix group names are identical to the
98 external VO names (as published in the CEAccessControlBaseRules),
99 then the vomap section is not needed. The only entries that are
100 needed here are those for which the external VO name is different
101 than the unix group name. Note that it is allowed to map several
102 unix group names onto a single "external" name, but each internal
103 group name must map to at most one external name.
104
105 The format looks like
106
107 vomap:
108 atlsgm : atlas
109 biome : biomed
110 LSF_lhcb : lhcb
111
112 The leading spaces are significant, they signify that each line is a
113 continuation of the 'vomap' parameter. The left hand of each line is
114 the unix group name, the right hand is the external VO name found in the
115 GlueCEAccessControlBaseRule field. If the unix group name and VO name
116 are the same, there is no need to include it in the vomap construct.
117 NOTE I have yet to test what happens if vomap is empty.
118
119 New in release 2.0 : the vomap can also map unix groups to VOMS FQANs,
120 like
121
122 aliceprd : /alice/Role=production
123
124 which maps the unix group 'aliceprd' to the listed VOMS FQAN. If
125 there is a VOView for this FQAN in the static_ldif_file, the relevant
126 info will be printed for jobs belonging to this unix group.
127
128 [LRMS] lrms_backend_cmd
129
130 Set this to a string that will run the command producing the queue
131 state information (first part of the lrms-specific part). This could
132 either be a real command like
133
134 /path/to/cmd/lrmsinfo-pbs -c /other/path/to/config_file_if_needed
135
136 or it could simply write the contents of some file to standard output
137 if you choose to generate the queue state information by a mechanism
138 other than the GIP subsystem:
139
140 cat /place/where/you/find/text_file_containing_the_generic_lrms_state.info
141
142 [Scheduler] vo_max_jobs_cmd
143
144 This is the same kind of critter as [LRMS] lrms_backend_cmd, except it
145 will provide for each unix group known to the LRMS, the maximum number
146 of job slots this group can take. Mapping from group to VO is handled
147 by the vomap parameter. If a VO has no cap on the number of jobs, it
148 can be left out. This entire command is optional, in which case the
149 free slots will only be limited by the number of free CPUs (or will be
150 set to zero when jobs for a VO are in a waiting state, in which case
151 there are obviously no free slots at the moment).
152
153 [Scheduler] cycle_time
154
155 There are various pieces of information that only make sense modulo
156 the scheduler cycle time. For example if the ERT prediction is
157 'zero', it is reset to half the scheduler cycle since the job will
158 almost never be run immediately, unless it happens to have been
159 submitted one second before the scheduling cycle starts. Set this
160 parameter to the length of your scheduling cycle. It is optional.
161 NOTE I have yet to test what happens if you leave this out.
162
163 LIMITATIONS
164
165 The program assumes that the structure of the static LDIF file is always
166 such that each block (corresponding to a given DN) is separated from the
167 next one by a blank line. If this is not the case, the parser will fail.
168
169 The program assumes that dn's it needs to parse are constructed
170 like
171
172 dn: GlueCEUniqueID=lxb...
173
174 where the line begins in the first column, and there is exactly one
175 space between the colon and the 'G' character.
176
177 For the purposes of VOView scheduling information, the plugin only
178 looks at the last AccessControlBaseRule in each VOView. The assumption
179 here is that the only reason that two ACBRs would be present in a single VOView
180 is if they mapped to the same underlying group in the batch system, in which
181 case it does not matter which one is used. The last one happens to be easier
182 for parsing purposes.

grid.support@nikhef.nl
ViewVC Help
Powered by ViewVC 1.1.28