ENGINE======With OpenSSL 0.9.6, a new component was added to support alternativecryptography implementations, most commonly for interfacing with externalcrypto devices (eg. accelerator cards). This component is called ENGINE,and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)caused a little confusion as 0.9.6** releases were rolled in twoversions, a "standard" and an "engine" version. In development for 0.9.7,the ENGINE code has been merged into the main branch and will be presentin the standard releases from 0.9.7 forwards.There are currently built-in ENGINE implementations for the followingcrypto devices:o Microsoft CryptoAPIo VIA Padlocko nCipher CHILIn addition, dynamic binding to external ENGINE implementations is nowprovided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"section below for details.At this stage, a number of things are still needed and are being worked on:1 Integration of EVP support.2 Configuration support.3 Documentation!1 With respect to EVP, this relates to support for ciphers and digests inthe ENGINE model so that alternative implementations of existingalgorithms/modes (or previously unimplemented ones) can be provided byENGINE implementations.2 Configuration support currently exists in the ENGINE API itself, in theform of "control commands". These allow an application to expose to theuser/admin the set of commands and parameter types a given ENGINEimplementation supports, and for an application to directly feed stringbased input to those ENGINEs, in the form of name-value pairs. This is anextensible way for ENGINEs to define their own "configuration" mechanismsthat are specific to a given ENGINE (eg. for a particular hardwaredevice) but that should be consistent across *all* OpenSSL-basedapplications when they use that ENGINE. Work is in progress (or at leastin planning) for supporting these control commands from the CONF (orNCONF) code so that applications using OpenSSL's existing configurationfile format can have ENGINE settings specified in much the same way.Presently however, applications must use the ENGINE API itself to providesuch functionality. To see first hand the types of commands availablewith the various compiled-in ENGINEs (see further down for dynamicENGINEs), use the "engine" openssl utility with full verbosity, ie;openssl engine -vvvv3 Documentation? Volunteers welcome! The source code is reasonably wellself-documenting, but some summaries and usage instructions are needed -moreover, they are needed in the same POD format the existing OpenSSLdocumentation is provided in. Any complete or incomplete contributionswould help make this happen.STABILITY & BUG-REPORTS=======================What already exists is fairly stable as far as it has been tested, butthe test base has been a bit small most of the time. For the most part,the vendors of the devices these ENGINEs support have contributed to thedevelopment and/or testing of the implementations, and *usually* (with noguarantees) have experience in using the ENGINE support to drive theirdevices from common OpenSSL-based applications. Bugs and/or inexplicablebehaviour in using a specific ENGINE implementation should be sent to theauthor of that implementation (if it is mentioned in the corresponding Cfile), and in the case of implementations for commercial hardwaredevices, also through whatever vendor support channels are available. Ifnone of this is possible, or the problem seems to be something about theENGINE API itself (ie. not necessarily specific to a particular ENGINEimplementation) then you should mail complete details to the relevantOpenSSL mailing list. For a definition of "complete details", refer tothe OpenSSL "README" file. As for which list to send it to;openssl-users: if you are *using* the ENGINE abstraction, either in anpre-compiled application or in your own application code.openssl-dev: if you are discussing problems with OpenSSL source code.USAGE=====The default "openssl" ENGINE is always chosen when performing cryptooperations unless you specify otherwise. You must actively tell theopenssl utility commands to use anything else through a new command lineswitch called "-engine". Also, if you want to use the ENGINE support inyour own code to do something similar, you must likewise explicitlyselect the ENGINE implementation you want.Depending on the type of hardware, system, and configuration, "settings"may need to be applied to an ENGINE for it to function as expected/hoped.The recommended way of doing this is for the application to supportENGINE "control commands" so that each ENGINE implementation can providewhatever configuration primitives it might require and the applicationcan allow the user/admin (and thus the hardware vendor's support deskalso) to provide any such input directly to the ENGINE implementation.This way, applications do not need to know anything specific to anydevice, they only need to provide the means to carry such user/admininput through to the ENGINE in question. Ie. this connects *you* (andyour helpdesk) to the specific ENGINE implementation (and device), andallows application authors to not get buried in hassle supportingarbitrary devices they know (and care) nothing about.A new "openssl" utility, "openssl engine", has been added in that allowsfor testing and examination of ENGINE implementations. Basic usageinstructions are available by specifying the "-?" command line switch.DYNAMIC ENGINES===============The new "dynamic" ENGINE provides a low-overhead way to support ENGINEimplementations that aren't pre-compiled and linked into OpenSSL-basedapplications. This could be because existing compiled-in implementationshave known problems and you wish to use a newer version with an existingapplication. It could equally be because the application (or OpenSSLlibrary) you are using simply doesn't have support for the ENGINE youwish to use, and the ENGINE provider (eg. hardware vendor) is providingyou with a self-contained implementation in the form of a shared-library.The other use-case for "dynamic" is with applications that wish tomaintain the smallest foot-print possible and so do not link in variousENGINE implementations from OpenSSL, but instead leaves you to providethem, if you want them, in the form of "dynamic"-loadableshared-libraries. It should be possible for hardware vendors to providetheir own shared-libraries to support arbitrary hardware to work withapplications based on OpenSSL 0.9.7 or later. If you're using anapplication based on 0.9.7 (or later) and the support you desire is onlyannounced for versions later than the one you need, ask the vendor tobackport their ENGINE to the version you need.How does "dynamic" work?------------------------The dynamic ENGINE has a special flag in its implementation such thatevery time application code asks for the 'dynamic' ENGINE, it in factgets its own copy of it. As such, multi-threaded code (or code thatmultiplexes multiple uses of 'dynamic' in a single application in anyway at all) does not get confused by 'dynamic' being used to do manyindependent things. Other ENGINEs typically don't do this so there isonly ever 1 ENGINE structure of its type (and reference counts are usedto keep order). The dynamic ENGINE itself provides absolutely nocryptographic functionality, and any attempt to "initialise" the ENGINEautomatically fails. All it does provide are a few "control commands"that can be used to control how it will load an external ENGINEimplementation from a shared-library. To see these control commands,use the command-line;openssl engine -vvvv dynamicThe "SO_PATH" control command should be used to identify theshared-library that contains the ENGINE implementation, and "NO_VCHECK"might possibly be useful if there is a minor version conflict and you(or a vendor helpdesk) is convinced you can safely ignore it."ID" is probably only needed if a shared-library implementsmultiple ENGINEs, but if you know the engine id you expect to be using,it doesn't hurt to specify it (and this provides a sanity check ifnothing else). "LIST_ADD" is only required if you actually wish theloaded ENGINE to be discoverable by application code later on using theENGINE's "id". For most applications, this isn't necessary - but someapplication authors may have nifty reasons for using it. The "LOAD"command is the only one that takes no parameters and is the commandthat uses the settings from any previous commands to actually *load*the shared-library ENGINE implementation. If this command succeeds, the(copy of the) 'dynamic' ENGINE will magically morph into the ENGINEthat has been loaded from the shared-library. As such, any controlcommands supported by the loaded ENGINE could then be executed as pernormal. Eg. if ENGINE "foo" is implemented in the shared-library"libfoo.so" and it supports some special control command "CMD_FOO", thefollowing code would load and use it (NB: obviously this code has noerror checking);ENGINE *e = ENGINE_by_id("dynamic");ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);For testing, the "openssl engine" utility can be useful for this sortof thing. For example the above code excerpt would achieve much thesame result as;openssl engine dynamic \-pre SO_PATH:/lib/libfoo.so \-pre ID:foo \-pre LOAD \-pre "CMD_FOO:some input data"Or to simply see the list of commands supported by the "foo" ENGINE;openssl engine -vvvv dynamic \-pre SO_PATH:/lib/libfoo.so \-pre ID:foo \-pre LOADApplications that support the ENGINE API and more specifically, the"control commands" mechanism, will provide some way for you to passsuch commands through to ENGINEs. As such, you would select "dynamic"as the ENGINE to use, and the parameters/commands you pass wouldcontrol the *actual* ENGINE used. Each command is actually a name-valuepair and the value can sometimes be omitted (eg. the "LOAD" command).Whilst the syntax demonstrated in "openssl engine" uses a colon toseparate the command name from the value, applications may providetheir own syntax for making that separation (eg. a win32 registrykey-value pair may be used by some applications). The reason for the"-pre" syntax in the "openssl engine" utility is that some commandsmight be issued to an ENGINE *after* it has been initialised for use.Eg. if an ENGINE implementation requires a smart-card to be insertedduring initialisation (or a PIN to be typed, or whatever), there may bea control command you can issue afterwards to "forget" the smart-cardso that additional initialisation is no longer possible. Inapplications such as web-servers, where potentially volatile code mayrun on the same host system, this may provide some arguable securityvalue. In such a case, the command would be passed to the ENGINE afterit has been initialised for use, and so the "-post" switch would beused instead. Applications may provide a different syntax forsupporting this distinction, and some may simply not provide it at all("-pre" is almost always what you're after, in reality).How do I build a "dynamic" ENGINE?----------------------------------This question is trickier - currently OpenSSL bundles various ENGINEimplementations that are statically built in, and any application thatcalls the "ENGINE_load_builtin_engines()" function will automaticallyhave all such ENGINEs available (and occupying memory). Applicationsthat don't call that function have no ENGINEs available like that andwould have to use "dynamic" to load any such ENGINE - but on the otherhand such applications would only have the memory footprint of anyENGINEs explicitly loaded using user/admin provided control commands.The main advantage of not statically linking ENGINEs and only using"dynamic" for hardware support is that any installation using no"external" ENGINE suffers no unnecessary memory footprint from unusedENGINEs. Likewise, installations that do require an ENGINE incur theoverheads from only *that* ENGINE once it has been loaded.Sounds good? Maybe, but currently building an ENGINE implementation asa shared-library that can be loaded by "dynamic" isn't automated inOpenSSL's build process. It can be done manually quite easily however.Such a shared-library can either be built with any OpenSSL code itneeds statically linked in, or it can link dynamically against OpenSSLif OpenSSL itself is built as a shared library. The instructions arethe same in each case, but in the former (statically linked anydependencies on OpenSSL) you must ensure OpenSSL is built withposition-independent code ("PIC"). The default OpenSSL compilation mayalready specify the relevant flags to do this, but you should consultwith your compiler documentation if you are in any doubt.This example will show building the "atalla" ENGINE in thecrypto/engine/ directory as a shared-library for use via the "dynamic"ENGINE.1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSLsource tree.2) Recompile at least one source file so you can see all the compilerflags (and syntax) being used to build normally. Eg;touch hw_atalla.c ; makewill rebuild "hw_atalla.o" using all such flags.3) Manually enter the same compilation line to compile the"hw_atalla.c" file but with the following two changes;(a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,(b) change the output file from "hw_atalla.o" to something new,eg. "tmp_atalla.o"4) Link "tmp_atalla.o" into a shared-library using the top-levelOpenSSL libraries to resolve any dependencies. The syntax for doingthis depends heavily on your system/compiler and is a nightmareknown well to anyone who has worked with shared-library portabilitybefore. 'gcc' on Linux, for example, would use the following syntax;gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto5) Test your shared library using "openssl engine" as explained in theprevious section. Eg. from the top-level directory, you might try;apps/openssl engine -vvvv dynamic \-pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOADIf the shared-library loads successfully, you will see both "-pre"commands marked as "SUCCESS" and the list of control commandsdisplayed (because of "-vvvv") will be the control commands for the*atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also addthe "-t" switch to the utility if you want it to try and initialisethe atalla ENGINE for use to test any possible hardware/driverissues.PROBLEMS========It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.A quick test done right before the release showed that trying "openssl speed-engine cswift" generated errors. If the DSO gets enabled, an attempt is madeto write at memory address 0x00000002.