/[pdpsoft]/trunk/grid-mw-security/ees/thesis/ees.tex
ViewVC logotype

Contents of /trunk/grid-mw-security/ees/thesis/ees.tex

Parent Directory Parent Directory | Revision Log Revision Log


Revision 953 - (show annotations) (download) (as text)
Wed Oct 21 08:40:48 2009 UTC (12 years, 3 months ago) by aramv
File MIME type: text/x-latex
File size: 18326 byte(s)
One to beam up
1 \chapter{The Execution Environment Service}
2 %The
3 % introduction to EES
4 The role of the \textit{Execution Environment Service} is to ensure that an appropriate site-specific execution environment is procured based on the site-agnostic obligations and assertions it receives as input in the form of SAML2XACML2 \cite{authzinterop} requests.
5 %It provides functionality similar to LCMAPS, but is never invoked directly through gLExec.
6 It aims to be a more flexible LCMAPS that can be directly interfaced with through SAML2XACML2.
7
8 Here is a description of the EES taken from the document "Design of the Execution Environment Service" \cite{ees_design} for EGEE.
9 \begin{quote}
10 \textit{
11 From the 'outside' the EES appears as an 'obligation transformer'.
12 It takes execution-agnostic assertions and obligations and makes sure that
13 \begin{itemize}
14 \item an execution environment compatible with these assertions and obligations is procured, e.g. by creating or leasing a Unix account;
15 \item new obligations are created that ensure that the policy enforcement points (PEP) can properly move the task into the procured execution environment;
16 \item removes any agnostic obligations that have been fully translated into a site-specific obligation, since the PEP has to be able to enforce all obligations present.
17 \end{itemize}
18 }
19 \end{quote}
20
21 \pagebreak
22 \section{Requirements}
23 These requirements were provided by developers of previous middleware services:
24
25 \subsection{Functional requirements}
26 The EES should \ldots
27 \begin{description}
28 \item [Be easily adaptable to new use cases]
29 {Future use cases that should be supported include interaction with virtualization frameworks or batch systems}
30 \item [Provide a data store for generic and specialized data types]{
31 It should be possible to let plug-ins exchange information by providing them with an API to a common data store}
32 \item [Have the ability to work with SAML2-XACML2 concepts]{
33 External services will interact with the EES through this protocol.
34 It should be able to interpret incoming requests and return a valid response.
35 }
36 \item [Facilitate access from several external interfaces]
37 {The EES should be a service that can handle communication from various external interfaces at a time to increase the flexibility of the service.
38 Please see section \ref{proposed_arch} for a list of possible interfaces.}
39 \end{description}
40
41 \subsection{Technical requirements}
42 The EES should \ldots
43 \begin{description}
44 \item [Be largely backward-compatible with existing deployment schemes]{
45 The plug-in API and configuration file format must be fully backward compatible with those of LCMAPS.
46 This will ease the transition from existing LCMAPS configurations and SCAS-based deployments to configurations where the EES will be used.
47 }
48 \item [Perform efficiently]{
49 Efficient performance is required to let the service handle many clients.
50 }
51 \item [Be thread-safe]{
52 Because the EES must support multiple concurrent users, its components must be reliable when used in concurrent threads.
53 }
54 \item [Be portable to many different platforms]{
55 %Grid sites commonly employ various Unix-like systems.
56 The hardware and software configuration of Grid systems that the EES must run on cannot be foreseen, however various standards \cite{Schildt:AAC90} \cite{IEEE:POSIX.1-90} exist that we can assume are supported.
57 %We should focus on a subset
58 %Assumptions about software configurations, installed libraries should be tested
59 %We can assume these adhere to standards,
60 %While these should adhere to the POSIX \cite{IEEE:POSIX.1-90} standard, there is a range of incompatibilities between different
61
62 }
63 \item [Be able to interact with the OS on a low level]{
64 To perform mapping of user credentials to Unix accounts, low-level OS interaction is required.
65 }
66 \item [Provide a flexible plug-in API]{
67 This API should enable the plug-ins to handle new use-cases.
68 }
69 \end{description}
70 % Moet de mogelijkheid hebben om low-level OS interacties te kunnen doen
71
72
73 \section{Design considerations}
74 \subsection{Programming language}
75 %Since performance and portability are a requirement the best contenders are C and C++.
76 %For the pu
77 What follows is a comparison of some popular languages known to be able to meet most of the technical requirements.
78 %I have weighed the technical and functional requirements against the pro's and cons I associate with each language.
79 %Some parts of this project seem like they might benefit from an Object Oriented approach.
80
81 \subsubsection{Java}
82 Other services in the Argus framework have been implemented in Java, so it might be possible to share experiences and code.
83 %However, since SAML2XACML2 has been agreed upon as a standard in which to exchange information so this is not a necessity to use Java.
84 It is renowned for its portability.
85 Although its performance has greatly caught up with the other contenders over the past decade, it is likely to be less efficient at managing memory.
86
87 Pros:
88 \begin{itemize}
89 \item Fully Object Oriented
90 \item Supposedly very portable
91 \end{itemize}
92 Cons:
93 \begin{itemize}
94 \item Not native to the OS, so hard to do low-level interaction
95 \item Likely to be a memory hog
96 %\item Hard to deploy stably
97 \end{itemize}
98
99 \subsubsection{C}
100 The previous middleware systems were created in C, which means there is code and experience at hand.
101 It is renowned for its performance.
102 While it does not natively provide any of the Object Oriented features the other languages do, a pseudo-OO style of programming can be adapted by using structures as simple Objects.
103 %Is a strong argument with regards to backward compatibility.
104 %This enables enforcing a strict separation and encapsulation of responsibilities.
105 %C is able to meet all of the technical requirements,
106
107 Pros:
108 \begin{itemize}
109 \item Offers good low-level OS interaction
110 \item Can offer good overall performance on most architectures
111 \item Has proven to be very portable
112 \end{itemize}
113 Cons:
114 \begin{itemize}
115 \item Requires careful programming
116 \item Sometimes a bit too low-level
117 \end{itemize}
118
119 \subsubsection{C++}
120 C++ can provide more functionality than C at the cost of some overhead and being less portable.
121 It is renowned for its efficiency at dealing with large data structures.
122 %The modern language constructs in C++ can complicate a simple design, which leads to a variety of issues.
123 Some of the conveniences provided by the use of the STL can lead to portability issues.
124
125 Pros:
126 \begin{itemize}
127 \item Offers good low-level OS interaction
128 \item Can offer good overall performance on most architectures
129 \end{itemize}
130 Cons:
131 \begin{itemize}
132 \item Requires careful programming
133 \item Modern programming style not as widely supported as C
134 \item Standard library doesn't adhere to a common standard between platforms
135 \end{itemize}
136
137 \subsubsection{Choice}
138 Java seems to be the worst option since it abstracts the low-level OS interaction required, and does so at the cost of efficiency.
139 I have chosen C because of the following reasons:
140
141 \begin{itemize}
142 \item The low-level OS interaction we seek is actually present in both C and C++
143 \item While C++ allows programming in an Object Oriented style it is much newer, not standardized and thus not as portable
144 \item C can perform slightly better because the overhead of OO constructs can be omitted
145 \item There is a lot of existing code and experience available
146 % TODO? wellicht description van maken?
147 \end{itemize}
148
149 \subsection{Implementation plan}
150 \subsubsection{Functionality}
151 The EES should be backwards compatible with the LCMAPS plug-in API and LCMAPS configuration file syntax.
152 LCMAPS already has a lot of the functionality required, but it is tightly coupled to existing plug-ins.
153 In particular the data structures present in LCMAPS are limited, and cannot be used to express arbitrary information.
154 Exposing an API that allows plug-ins to exchange arbitrary information with each other through a data store is a way to decouple the service logic from the Grid-specific details needed to enforce the configured policies.
155
156 To assure forward compatibility, the EES should ideally be agnostic about Grid-related business logic, instead relying on configurable chains of plug-ins to contain this.
157 By linking together small, maintainable plug-ins that can interact with the data store and perform a variety of tasks, the system can support many use cases.
158 \label{pipe_mechanism}
159 This is analogous to the concept of pipes on Unix systems, which can chain together multiple processes to allow them to interact with each other.
160
161 Because all the Grid-specific logic can be encapsulated in plug-ins, the requirements allow the core of the EES to be almost completely oblivious to anything specifically Grid-related.
162
163 \subsubsection{Reuse of legacy code}
164 %Several components of LCMAPS are
165 The LCMAPS framework roughly consists of the following components:
166
167 \begin{description}
168 \item[Evaluation manager]{
169 Parses the configuration file to load configuration of plug-ins and authorization policies.
170 }
171 \item[Plug-in manager]{
172 Loads and executes plug-ins configured by the Evaluation Manager.
173 }
174 \item[Control logic]{
175 These functions control the execution of the framework.
176 }
177 \item[Library functions]{
178 Library functions dealing with Grid-specific business logic, to be used by plug-ins.
179 }
180 \item[Plug-ins]{
181 Plug-ins make use of the library functions to enforce configured authorization policies.
182 }
183 \end{description}
184
185 Based on the functionality proposed for the EES, I assess the following elements can be reused:
186
187 \begin{description}
188 \item[Data structures]{
189 The data structures that the plug-in manager and evaluation manager use to keep track of plug-ins and policies might be reused, as the same mechanisms must be facilitated.
190 The implementation of the logic that interacts with these data structures will certainly be different, but will likely use them for the same mechanisms.
191 }
192 \item[Configuration file parser]{
193 The LCMAPS parser is written in Lex and Yacc and hooks into the framework by calling functions present in the evaluation manager.
194 Because the Lex/Yacc parser is a relatively loosely coupled component, it is likely possible to reuse this code in the EES.
195 The callback functions used by the parser in LCMAPS will likely have to be reimplemented due to their strong coupling to the rest of the LCMAPS framework.
196 }
197 \item[Plug-ins]{
198 Ideally the LCMAPS plug-ins can be used with in EES after applying minimal patches that enable use of the AOS.
199 }
200 \item[Use of libraries]{
201 The SCAS holds logic to parse SAML2XACML2 statements.
202 This logic can likely be reused in the EES when adapted to interact with the data store.
203 }
204 \end{description}
205
206 \subsubsection{Portability}
207 To ensure portability to many Unix platforms the software should conform to the ANSI C \cite{Schildt:AAC90} and POSIX \cite{IEEE:POSIX.1-90} standards.
208 The use of GNU extensions to the C language is normally not portable, but these can be provided by the GNU Gnulib \cite{gnulib} portability library.
209
210 By using GNU Autotools \cite{automake} \cite{autoconf} a configuration script can be created from a manageable configuration file.
211 The use of GNU M4 macros \cite{m4} allows developers to create a feature-rich script, with which for example the path to plug-in modules can be specified.
212
213 \subsubsection{Service and library programming}
214 Because multiple interfaces must be supported, it makes sense to encapsulate the core logic of the software apart from these interfaces.
215 Most of the implementation time will be spent on this core logic, allowing it to be incorporated into a service at a later time.
216
217 \subsubsection{APIs}
218 There are two kinds of APIs that must be exposed from the core framework:
219 \begin{itemize}
220 \item An API for the interface components to interact with the core framework
221 \item An API for plug-ins to interact with the library functions in the core framework
222 \end{itemize}
223
224 The plug-in API is more important to get right, because more developers will be depending on it.
225
226 %This design requires a clear and concise API, most notably for plug-ins.
227 \subsubsection{Thread-safety}
228 To ensure thread-safety it should suffice to initialize a separate data store for each request, as this data is transient.
229 The rest of the configuration of the service is static.
230 %To ease the creation of services that access the core functionality, I was advised to create a library that exposes it.
231 %This has the added benefit of separating the core logic of the service from the interface components.
232
233 % Maken van kleine, aan elkaar geknoopte dingetjes in C is minder verneukeratief (functie scheiding)
234
235 %Particularly the parser
236
237 %The project can largely be based on LCMAPS, as it provides most of the functionality that will eventually be required in the EES.
238 %However, due to the age and relative complexity of the LCMAPS codebase I have chosen to start this project from scratch, borrowing code where applicable.
239
240
241
242 \glossary{name={EES}, description={Execution Environment Service}}
243 \nomenclature{EES}{Execution Environment Service}
244
245 \subsection{Proposed architecture for the EES}
246 \label{proposed_arch}
247 The core component of the \textit{Execution Environment Service} (EES) is a shared library that exposes the core logic, hereby dubbed the \textit{EES Execution Framework} (EEF).
248 \glossary{name={EEF}, description={EES Execution Framework}}
249 \nomenclature{EEF}{EES Execution Framework}
250
251 The EES provides access to the EEF through various mechanisms, called \textit{EES Interfaces} (EIs), which are implemented as \textit{EES Interface Components} (EICs).
252
253 These EICs can be implemented as (for example):
254 \begin{itemize}
255 \item An HTTP(S) socket
256 \item A Unix domain socket
257 \item A named pipe
258 \end{itemize}
259
260 Multiple EICs must be supported so that many kinds of internal and external services can interact with the EES.
261 Requests travel from these EICs through the EEF framework to plug-ins, that do the actual work.
262 This design was chosen to clearly separate the roles of the EEF and the EICs, thereby encapsulating the core logic away from disparate interfaces.
263 %redenen voor deze EIC's.
264 % meerdere interaces zodat allerlei soorten services ermee kunnen functioneren (intern en extern)
265
266 % Vanaf de EIC's ga je door de EEF heen naar de plug-ins.
267
268 \glossary{name={EIC}, description={EES Interface Component}}
269 \nomenclature{EIC}{EES Interface Component}
270
271 Besides using these EICs a direct API invocation of the EEF is also possible.
272
273
274 % EEF is agnostisch, dom ding. Daarom geschikt om pluggable te maken.
275
276 %\begin{figure}[hp]
277 %\centering
278 %\includegraphics[width=\textwidth]{ees_class_diagram}
279 %\caption[Pseudo-class diagram of the EES execution framework]%
280 %{Pseudo-class diagram of the EES Execution Framework, which shows the different files associated with different roles in the design.}
281 %\end{figure}
282
283
284 Please see appendix \ref{ees_class_diagram} for a diagram showing the architecture of the EES, which also describes the internal EEF components.
285
286 \section{Proposed components of the EEF}
287 The architecture of the EEF is similar to that of LCMAPS, with the addition of the AOS.
288 The library functions present in LCMAPS can be contained in plug-ins through use of the 'pipe' mechanism described in section \ref{pipe_mechanism}.
289
290 \subsection{Evaluation Manager (EM)}
291 \glossary{name={EM}, description={Evaluation Manager}}
292 The \textit{Evaluation Manager} parses the configuration file and loads configuration of specified plug-ins.
293 Once the plug-in descriptions have been initialized, the plug-in chains expressed in the configuration file should be created.
294 %These chains serve to describe different paths to successful execution of the plug-ins, which might fail at any point in the chain.
295 The configuration should be checked to avoid creating recursions in the policy chains.
296 After the configuration has been successfully parsed the Plug-in Manager can be used to load the specified plug-ins.
297
298 \subsection{Plug-in Manager (PM)}
299 \glossary{name={PM}, description={Plug-in Manager}}
300 \nomenclature{PM}{Plug-in Manager}
301 The Plug-in Manager, controlled by the Evaluation Manager, creates a list of plug-ins and is provides functions to initialize, run and terminate them.
302 The list is built up of structures similar to those used in LCMAPS.
303 % TODO hoe en waarom
304
305 \subsection{Attribute and Obligation Store (AOS)}
306 \glossary{name={AOS}, description={Attribute \& Obligation Store}}
307 \nomenclature{AOS}{Attribute \& Obligation Store}
308 The \textit{Attribute and Obligation Store} is used to store and exchange data between plug-ins.
309 Each data element should be uniquely identified through a label, and hold a copy of the supplied data in memory.
310 Plug-ins should not be able to overwrite or delete each other's data.
311
312 Data can be retrieved from the AOS by its label through getter functions which return type-casted results.
313 This way, a crude mechanism for dynamic typing has been implemented.
314 %Ownership of the stored data could be established through the memory address of the setting plug-in.
315 % TODO stating the rules is not a motivation for its design
316 % TODO needs more words! :)
317
318 %\subsection{Tools}
319 %At least some logging /debugging functionality should be implemented.
320
321 \pagebreak
322 \section{Life-cycle of an EEF run}
323 \begin{itemize}
324 % open logfile, AOS init, eef_run voor request
325 \item The EEF is invoked with a string containing a path to the LCMAPS-compatible policy description file.
326 \item To enable plug-ins to exchange information while being executed consecutively the AOS is initialized by the EEF
327 \item The EEF then invokes the Evaluation Manager to parse the supplied policy file, which creates a list of plug-in chains that can lead to successful execution.
328 \item The Evaluation Manager then loads and initializes the specified plug-ins by calling the Plug-in Manager
329 \item If all the plug-ins have been successfully loaded they are ready to be executed by a call to the EEF
330 \item Once a request comes in through an EIC, the SAML statements are parsed and loaded into the AOS
331 \item The policy chains loaded from the configuration file are run until one chain completes successfully
332 \item After completion, the result is returned to the EIC
333 \end{itemize}
334
335 %hier wordt het juist interessant
336 % Beschrijf:
337 % state-machine
338 % AOS interactie
339 % afhandeling van failure mbt AOS state
340 % cleanup & terminate van alle plug-ins
341
342 Please see appendix \ref{ees_sequence_diagram} for a sequence diagram of the EEF.
343

grid.support@nikhef.nl
ViewVC Help
Powered by ViewVC 1.1.28