/[pdpsoft]/trunk/grid-mw-security/ees/thesis/ees.tex
ViewVC logotype

Diff of /trunk/grid-mw-security/ees/thesis/ees.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 952 by aramv, Tue Oct 20 14:33:07 2009 UTC revision 953 by aramv, Wed Oct 21 08:40:48 2009 UTC
# Line 1  Line 1 
1  \chapter{The Execution Environment Service}  \chapter{The Execution Environment Service}
2  %The  %The
3  % introduction to EES  % introduction to EES
4  The role of the \textit{Execution Environment Service} is to ensure that an appropriate site-specific execution environment is procured based on the site-agnostic obligations and assertions it receives as input in the form of SAML2-XACML2 \cite{authzinterop} requests.  The role of the \textit{Execution Environment Service} is to ensure that an appropriate site-specific execution environment is procured based on the site-agnostic obligations and assertions it receives as input in the form of SAML2XACML2 \cite{authzinterop} requests.
5  It provides functionality similar to the SCAS, but delegates the enforcement to the PEP.  %It provides functionality similar to LCMAPS, but is never invoked directly through gLExec.
6    It aims to be a more flexible LCMAPS that can be directly interfaced with through SAML2XACML2.
7    
8  Here is a description of the EES taken from the document "Description of the EES" \cite{ees_design} for EGEE.  Here is a description of the EES taken from the document "Design of the Execution Environment Service" \cite{ees_design} for EGEE.
9  \begin{quote}  \begin{quote}
10  \textit{  \textit{
11  From the ‘outside’ the EES appears as an ‘obligation transformer’.  From the 'outside' the EES appears as an 'obligation transformer'.
12  It takes execution-agnostic assertions and obligations and makes sure that  It takes execution-agnostic assertions and obligations and makes sure that
13  \begin{itemize}  \begin{itemize}
14  \item an execution environment compatible with these assertions and obligations is procured, e.g. by creating or leasing a Unix account;  \item an execution environment compatible with these assertions and obligations is procured, e.g. by creating or leasing a Unix account;
# Line 17  It takes execution-agnostic assertions a Line 18  It takes execution-agnostic assertions a
18  }  }
19  \end{quote}  \end{quote}
20    
21    \pagebreak
22  \section{Requirements}  \section{Requirements}
23    These requirements were provided by developers of previous middleware services:
24    
25  \subsection{Functional requirements}  \subsection{Functional requirements}
26  The EES should \ldots  The EES should \ldots
27  \begin{itemize}  \begin{description}
28  \item be easily adaptable to new use cases, such as those pertaining to virtualization frameworks or interactions with batch systems  \item [Be easily adaptable to new use cases]
29  \item facilitate access from several external interfaces  {Future use cases that should be supported include interaction with virtualization frameworks or batch systems}
30  \item provide a data store for generic and specialized data types  \item [Provide a data store for generic and specialized data types]{
31  \item have a mechanism to work with SAML2-XACML2 concepts  It should be possible to let plug-ins exchange information by providing them with an API to a common data store}
32  \end{itemize}  \item [Have the ability to work with SAML2-XACML2 concepts]{
33    External services will interact with the EES through this protocol.
34    It should be able to interpret incoming requests and return a valid response.
35    }
36    \item [Facilitate access from several external interfaces]
37    {The EES should be a service that can handle communication from various external interfaces at a time to increase the flexibility of the service.
38    Please see section \ref{proposed_arch} for a list of possible interfaces.}
39    \end{description}
40    
41  \subsection{Technical requirements}  \subsection{Technical requirements}
42  The EES should \ldots  The EES should \ldots
43  \begin{itemize}  \begin{description}
44  \item be largely backward-compatible with existing deployment schemes (provided the required plug-ins are updated accordingly)  \item [Be largely backward-compatible with existing deployment schemes]{
45  \item be thread-safe  The plug-in API and configuration file format must be fully backward compatible with those of LCMAPS.
46  \item perform efficiently  This will ease the transition from existing LCMAPS configurations and SCAS-based deployments to configurations where the EES will be used.
47  \item be portable to many different platforms  }
48  \item be able to interact with the OS on a low level  \item [Perform efficiently]{
49  \item provide a flexible plug-in API  Efficient performance is required to let the service handle many clients.
50  \end{itemize}  }
51    \item [Be thread-safe]{
52    Because the EES must support multiple concurrent users, its components must be reliable when used in concurrent threads.
53    }
54    \item [Be portable to many different platforms]{
55    %Grid sites commonly employ various Unix-like systems.
56    The hardware and software configuration of Grid systems that the EES must run on cannot be foreseen, however various standards \cite{Schildt:AAC90} \cite{IEEE:POSIX.1-90} exist that we can assume are supported.
57    %We should focus on a subset
58    %Assumptions about software configurations, installed libraries should be tested
59    %We can assume these adhere to standards,
60    %While these should adhere to the POSIX \cite{IEEE:POSIX.1-90} standard, there is a range of incompatibilities between different
61    
62    }
63    \item [Be able to interact with the OS on a low level]{
64    To perform mapping of user credentials to Unix accounts, low-level OS interaction is required.
65    }
66    \item [Provide a flexible plug-in API]{
67    This API should enable the plug-ins to handle new use-cases.
68    }
69    \end{description}
70  % Moet de mogelijkheid hebben om low-level OS interacties te kunnen doen  % Moet de mogelijkheid hebben om low-level OS interacties te kunnen doen
71    
72    
# Line 54  Other services in the Argus framework ha Line 84  Other services in the Argus framework ha
84  It is renowned for its portability.  It is renowned for its portability.
85  Although its performance has greatly caught up with the other contenders over the past decade, it is likely to be less efficient at managing memory.  Although its performance has greatly caught up with the other contenders over the past decade, it is likely to be less efficient at managing memory.
86    
87  Pro's:  Pros:
88  \begin{itemize}  \begin{itemize}
89  \item Fully Object Oriented  \item Fully Object Oriented
90  \item Supposedly very portable  \item Supposedly very portable
91  \end{itemize}  \end{itemize}
92  Cons:  Cons:
93  \begin{itemize}  \begin{itemize}
94  \item Not native, so hard to do low-level OS interaction  \item Not native to the OS, so hard to do low-level interaction
95  \item Likely to be a memory hog  \item Likely to be a memory hog
96  %\item Hard to deploy stably  %\item Hard to deploy stably
97  \end{itemize}  \end{itemize}
# Line 74  While it does not natively provide any o Line 104  While it does not natively provide any o
104  %This enables enforcing a strict separation and encapsulation of responsibilities.  %This enables enforcing a strict separation and encapsulation of responsibilities.
105  %C is able to meet all of the technical requirements,  %C is able to meet all of the technical requirements,
106    
107  Pro's:  Pros:
108  \begin{itemize}  \begin{itemize}
109  \item Offers good low-level OS interaction  \item Offers good low-level OS interaction
110  \item Can offer good overall performance on most architectures  \item Can offer good overall performance on most architectures
# Line 92  It is renowned for its efficiency at dea Line 122  It is renowned for its efficiency at dea
122  %The modern language constructs in C++ can complicate a simple design, which leads to a variety of issues.  %The modern language constructs in C++ can complicate a simple design, which leads to a variety of issues.
123  Some of the conveniences provided by the use of the STL can lead to portability issues.  Some of the conveniences provided by the use of the STL can lead to portability issues.
124    
125  Pro's:  Pros:
126  \begin{itemize}  \begin{itemize}
127  \item Offers good low-level OS interaction  \item Offers good low-level OS interaction
128  \item Can offer good overall performance on most architectures  \item Can offer good overall performance on most architectures
129  \end{itemize}  \end{itemize}
130  Cons:  Cons:
131  \begin{itemize}  \begin{itemize}
132    \item Requires careful programming
133  \item Modern programming style not as widely supported as C  \item Modern programming style not as widely supported as C
134  \item Standard library doesn't adhere to a common standard between platforms  \item Standard library doesn't adhere to a common standard between platforms
135  \end{itemize}  \end{itemize}
136    
137  \subsubsection{Choice}  \subsubsection{Choice}
138  The low-level OS interaction we seek is actually present in both C and C++.  Java seems to be the worst option since it abstracts the low-level OS interaction required, and does so at the cost of efficiency.
139  I have chosen C because of the following reasons:  I have chosen C because of the following reasons:
140    
141  \begin{itemize}  \begin{itemize}
142  \item Java seems to be the worst option since it abstracts the low-level OS interaction required, and does so at the cost of efficiency  \item The low-level OS interaction we seek is actually present in both C and C++
143  \item While C++ allows programming in an Object Oriented style it is much newer, not standardized and thus not as portable  \item While C++ allows programming in an Object Oriented style it is much newer, not standardized and thus not as portable
144  \item C can perform slightly better because the overhead of OO constructs can be omitted  \item C can perform slightly better because the overhead of OO constructs can be omitted
145    \item There is a lot of existing code and experience available
146    % TODO? wellicht description van maken?
147  \end{itemize}  \end{itemize}
148    
149  \subsection{Implementation plan}  \subsection{Implementation plan}
150  \subsubsection{Functionality}  \subsubsection{Functionality}
151  To assure forward compatibility, the EES should be agnostic about low-level Grid business logic, instead relying on configurable plug-in chains to contain this.  The EES should be backwards compatible with the LCMAPS plug-in API and LCMAPS configuration file syntax.
152  This can be made more powerful by exposing an API which lets plug-ins exchange arbitrary information with each other through a data store.  LCMAPS already has a lot of the functionality required, but it is tightly coupled to existing plug-ins.
153    In particular the data structures present in LCMAPS are limited, and cannot be used to express arbitrary information.
154    Exposing an API that allows plug-ins to exchange arbitrary information with each other through a data store is a way to decouple the service logic from the Grid-specific details needed to enforce the configured policies.
155    
156    To assure forward compatibility, the EES should ideally be agnostic about Grid-related business logic, instead relying on configurable chains of plug-ins to contain this.
157  By linking together small, maintainable plug-ins that can interact with the data store and perform a variety of tasks, the system can support many use cases.  By linking together small, maintainable plug-ins that can interact with the data store and perform a variety of tasks, the system can support many use cases.
158  The requirements allow the core of the EES to be almost completely oblivious to anything specifically Grid-related.  \label{pipe_mechanism}
159    This is analogous to the concept of pipes on Unix systems, which can chain together multiple processes to allow them to interact with each other.
160    
161  \subsubsection{Service programming}  Because all the Grid-specific logic can be encapsulated in plug-ins, the requirements allow the core of the EES to be almost completely oblivious to anything specifically Grid-related.
162    
163    \subsubsection{Reuse of legacy code}
164    %Several components of LCMAPS are
165    The LCMAPS framework roughly consists of the following components:
166    
167    \begin{description}
168    \item[Evaluation manager]{
169    Parses the configuration file to load configuration of plug-ins and authorization policies.
170    }
171    \item[Plug-in manager]{
172    Loads and executes plug-ins configured by the Evaluation Manager.
173    }
174    \item[Control logic]{
175    These functions control the execution of the framework.
176    }
177    \item[Library functions]{
178    Library functions dealing with Grid-specific business logic, to be used by plug-ins.
179    }
180    \item[Plug-ins]{
181    Plug-ins make use of the library functions to enforce configured authorization policies.
182    }
183    \end{description}
184    
185    Based on the functionality proposed for the EES, I assess the following elements can be reused:
186    
187    \begin{description}
188    \item[Data structures]{
189    The data structures that the plug-in manager and evaluation manager use to keep track of plug-ins and policies might be reused, as the same mechanisms must be facilitated.
190    The implementation of the logic that interacts with these data structures will certainly be different, but will likely use them for the same mechanisms.
191    }
192    \item[Configuration file parser]{
193    The LCMAPS parser is written in Lex and Yacc and hooks into the framework by calling functions present in the evaluation manager.
194    Because the Lex/Yacc parser is a relatively loosely coupled component, it is likely possible to reuse this code in the EES.
195    The callback functions used by the parser in LCMAPS will likely have to be reimplemented due to their strong coupling to the rest of the LCMAPS framework.
196    }
197    \item[Plug-ins]{
198    Ideally the LCMAPS plug-ins can be used with in EES after applying minimal patches that enable use of the AOS.
199    }
200    \item[Use of libraries]{
201    The SCAS holds logic to parse SAML2XACML2 statements.
202    This logic can likely be reused in the EES when adapted to interact with the data store.
203    }
204    \end{description}
205    
206    \subsubsection{Portability}
207    To ensure portability to many Unix platforms the software should conform to the ANSI C \cite{Schildt:AAC90} and POSIX \cite{IEEE:POSIX.1-90} standards.
208    The use of GNU extensions to the C language is normally not portable, but these can be provided by the GNU Gnulib \cite{gnulib} portability library.
209    
210    By using GNU Autotools \cite{automake} \cite{autoconf} a configuration script can be created from a manageable configuration file.
211    The use of GNU M4 macros \cite{m4} allows developers to create a feature-rich script, with which for example the path to plug-in modules can be specified.
212    
213    \subsubsection{Service and library programming}
214  Because multiple interfaces must be supported, it makes sense to encapsulate the core logic of the software apart from these interfaces.  Because multiple interfaces must be supported, it makes sense to encapsulate the core logic of the software apart from these interfaces.
215  Most of the implementation time will be spent on this core logic, allowing it to be incorporated into a service at a later time.  Most of the implementation time will be spent on this core logic, allowing it to be incorporated into a service at a later time.
216    
217  \subsubsection{Legacy code}  \subsubsection{APIs}
218  Some of the data structures and code from LCMAPS can be reused in the EES.  There are two kinds of APIs that must be exposed from the core framework:
 Reusing the configuration file parser can be a quick win.  
   
 \subsubsection{API's}  
 There are two kinds of API's that must be exposed from the core framework:  
219  \begin{itemize}  \begin{itemize}
220  \item An API for the interface components to interact with the core framework  \item An API for the interface components to interact with the core framework
221  \item An API for plug-ins to interact with the core framework and the outside world  \item An API for plug-ins to interact with the library functions in the core framework
222  \end{itemize}  \end{itemize}
223    
224  The plug-in API is more important to get right, because more developers will be depending on it.  The plug-in API is more important to get right, because more developers will be depending on it.
# Line 142  The plug-in API is more important to get Line 227  The plug-in API is more important to get
227  \subsubsection{Thread-safety}  \subsubsection{Thread-safety}
228  To ensure thread-safety it should suffice to initialize a separate data store for each request, as this data is transient.  To ensure thread-safety it should suffice to initialize a separate data store for each request, as this data is transient.
229  The rest of the configuration of the service is static.  The rest of the configuration of the service is static.
 %TODO tijd om te gaan slapen  
230  %To ease the creation of services that access the core functionality, I was advised to create a library that exposes it.  %To ease the creation of services that access the core functionality, I was advised to create a library that exposes it.
231  %This has the added benefit of separating the core logic of the service from the interface components.  %This has the added benefit of separating the core logic of the service from the interface components.
232    
# Line 159  The rest of the configuration of the ser Line 243  The rest of the configuration of the ser
243  \nomenclature{EES}{Execution Environment Service}  \nomenclature{EES}{Execution Environment Service}
244    
245  \subsection{Proposed architecture for the EES}  \subsection{Proposed architecture for the EES}
246    \label{proposed_arch}
247  The core component of the \textit{Execution Environment Service} (EES) is a shared library that exposes the core logic, hereby dubbed the \textit{EES Execution Framework} (EEF).  The core component of the \textit{Execution Environment Service} (EES) is a shared library that exposes the core logic, hereby dubbed the \textit{EES Execution Framework} (EEF).
248  \glossary{name={EEF}, description={EES Execution Framework}}  \glossary{name={EEF}, description={EES Execution Framework}}
249  \nomenclature{EEF}{EES Execution Framework}  \nomenclature{EEF}{EES Execution Framework}
250    
251  The EES provides access to the EEF through various mechanisms, called \textit{EES Interfaces} (EI's), which are implemented as \textit{EES Interface Components} (EIC's).  The EES provides access to the EEF through various mechanisms, called \textit{EES Interfaces} (EIs), which are implemented as \textit{EES Interface Components} (EICs).
252    
253  These EIC's can be implemented as (for example):  These EICs can be implemented as (for example):
254  \begin{itemize}  \begin{itemize}
255  \item An HTTP(S) socket  \item An HTTP(S) socket
256  \item A Unix domain socket  \item A Unix domain socket
257  \item A named pipe  \item A named pipe
258  \end{itemize}  \end{itemize}
259    
260  Multiple interaction mechanisms must be supported so that many kinds of internal and external services can interact with the EES.  Multiple EICs must be supported so that many kinds of internal and external services can interact with the EES.
261  Requests travel from these EIC's through the EEF framework to plug-ins, that do the actual work.  Requests travel from these EICs through the EEF framework to plug-ins, that do the actual work.
262  This design was chosen to clearly separate the roles of the EEF and the EIC's, thereby encapsulating the core logic away from disparate interfaces.  This design was chosen to clearly separate the roles of the EEF and the EICs, thereby encapsulating the core logic away from disparate interfaces.
263  %redenen voor deze EIC's.  %redenen voor deze EIC's.
264  % meerdere interaces zodat allerlei soorten services ermee kunnen functioneren (intern en extern)  % meerdere interaces zodat allerlei soorten services ermee kunnen functioneren (intern en extern)
265    
# Line 183  This design was chosen to clearly separa Line 268  This design was chosen to clearly separa
268  \glossary{name={EIC}, description={EES Interface Component}}  \glossary{name={EIC}, description={EES Interface Component}}
269  \nomenclature{EIC}{EES Interface Component}  \nomenclature{EIC}{EES Interface Component}
270    
271  Besides using these EIC's a direct API invocation of the EEF is also possible.  Besides using these EICs a direct API invocation of the EEF is also possible.
272    
273    
274  % EEF is agnostisch, dom ding. Daarom geschikt om pluggable te maken.  % EEF is agnostisch, dom ding. Daarom geschikt om pluggable te maken.
# Line 199  Besides using these EIC's a direct API i Line 284  Besides using these EIC's a direct API i
284  Please see appendix \ref{ees_class_diagram} for a diagram showing the architecture of the EES, which also describes the internal EEF components.  Please see appendix \ref{ees_class_diagram} for a diagram showing the architecture of the EES, which also describes the internal EEF components.
285    
286  \section{Proposed components of the EEF}  \section{Proposed components of the EEF}
287    The architecture of the EEF is similar to that of LCMAPS, with the addition of the AOS.
288    The library functions present in LCMAPS can be contained in plug-ins through use of the 'pipe' mechanism described in section \ref{pipe_mechanism}.
289    
290  \subsection{Evaluation Manager (EM)}  \subsection{Evaluation Manager (EM)}
291  \glossary{name={EM}, description={Evaluation Manager}}  \glossary{name={EM}, description={Evaluation Manager}}
292  The \textit{Evaluation Manager} parses the configuration file and loads specified plug-ins.  The \textit{Evaluation Manager} parses the configuration file and loads configuration of specified plug-ins.
293  Once the plug-in descriptions have been initialized, the plug-in chains expressed in the configuration file should be created.  Once the plug-in descriptions have been initialized, the plug-in chains expressed in the configuration file should be created.
294  These chains serve to describe different paths to successful execution of the plug-ins, which might fail at any point in the chain.  %These chains serve to describe different paths to successful execution of the plug-ins, which might fail at any point in the chain.
295  The configuration should be checked to avoid creating recursions in the policy chains.  The configuration should be checked to avoid creating recursions in the policy chains.
296  After the configuration has been successfully parsed the Plug-in Manager can be used to loads the specified plug-ins.  After the configuration has been successfully parsed the Plug-in Manager can be used to load the specified plug-ins.
297    
298  \subsection{Plug-in Manager (PM)}  \subsection{Plug-in Manager (PM)}
299  \glossary{name={PM}, description={Plug-in Manager}}  \glossary{name={PM}, description={Plug-in Manager}}
300  \nomenclature{PM}{Plug-in Manager}  \nomenclature{PM}{Plug-in Manager}
301  The Plug-in Manager builds and maintains a list of plug-ins and is able to initialize, run and terminate them.  The Plug-in Manager, controlled by the Evaluation Manager, creates a list of plug-ins and is provides functions to initialize, run and terminate them.
302  The list is built up of structures that contain function pointers linked to functions in the actual plug-in.  The list is built up of structures similar to those used in LCMAPS.
 It is controlled by the Evaluation Manager.  
303  % TODO hoe en waarom  % TODO hoe en waarom
304    
305  \subsection{Attribute and Obligation Store (AOS)}  \subsection{Attribute and Obligation Store (AOS)}
306  \glossary{name={AOS}, description={Attribute \& Obligation Store}}  \glossary{name={AOS}, description={Attribute \& Obligation Store}}
307  \nomenclature{AOS}{Attribute \& Obligation Store}  \nomenclature{AOS}{Attribute \& Obligation Store}
308  The \textit{Attribute and Obligation Store} is used to store and exchange data between plug-ins.  The \textit{Attribute and Obligation Store} is used to store and exchange data between plug-ins.
 %Data can be retrieved from the AOS by its label through getter functions which return type-casted results.  
 %This way, a crude mechanism for dynamic typing has been implemented.  
309  Each data element should be uniquely identified through a label, and hold a copy of the supplied data in memory.  Each data element should be uniquely identified through a label, and hold a copy of the supplied data in memory.
310  Plug-ins should not be able to overwrite or delete each other's data.  Plug-ins should not be able to overwrite or delete each other's data.
311    
312    Data can be retrieved from the AOS by its label through getter functions which return type-casted results.
313    This way, a crude mechanism for dynamic typing has been implemented.
314  %Ownership of the stored data could be established through the memory address of the setting plug-in.  %Ownership of the stored data could be established through the memory address of the setting plug-in.
315  % TODO stating the rules is not a motivation for its design  % TODO stating the rules is not a motivation for its design
316    % TODO needs more words! :)
317    
318  %\subsection{Tools}  %\subsection{Tools}
319  %At least some logging /debugging functionality should be implemented.  %At least some logging /debugging functionality should be implemented.

Legend:
Removed from v.952  
changed lines
  Added in v.953

grid.support@nikhef.nl
ViewVC Help
Powered by ViewVC 1.1.28