GENIE: A general model-independent TCP/IP run manager Active

GENIE is a model-independent suite of programs that can be used to generally distribute, manage, and execute multiple model runs via a TCP/IP network. The suite consists of a file distribution interface, a run manager, a run executer, and a routine that can be compiled as part of a program and used to exchange model runs with the run manager. Because communication is via a standard protocol (TCP/IP), any computer connected to the Internet can serve in any of the capabilities offered by this suite. Model independence is consistent with the existing template and instruction file protocols of the widely used PEST parameter estimation suite.

VERISON HISTORY

Version 2 (CURRENT) - September 18, 2015

Version 1 - April 18, 2012

GENIE FAQs

GENIE Version 2 Functionality
What does GENIE do?
GENIE is designed to allow parallel access to networked computing resources. It can be used to generally distribute, manage, and execute multiple model runs via TCP/IP networks. This functionality is currently available for PPEST (Doherty, 2010) and PEST++ (Welter and others, 2015). In addition to the run manager and run executer that are of most use to the majority of users, the full suite also includes a GENIE_INTERFACE routine that can be compiled as part of any program and used to exchange model runs with the run manager. Because communication is via TCP/IP, any computer connected to the Internet can serve in any of the capacities offered by this suite. Model independence is consistent with the existing template and instruction file protocols of the widely used PEST program. The www.pesthomepage.org description of GENIE can be found here.

Can GENIE make my non-parallel code run in parallel across a network?
Perhaps yes, if the serial runs are "embarrassingly parallel" (numerical modeling problems characterized by multiple independent calculations with little communication needed between the independent calculations). The GENIE_INTERFACE provides a conduit for information regarding the exchange of model runs and associated results with the GENIE manager for an outside or calling program. The outside program needs to prepare the model run information in the array format required by GENIE and then needs to call the GENIE_INTERFACE routine. This interface routine allows dynamic two-way communication between the outside program and the GENIE manager. The routine is written in C++ but contains the necessary external interface for compilation with Fortran. Version 2 of the GENIE_INTERFACE is described in Muffels and others (2015), Appendix 2 of the PEST++ Version 3 report.

What are some of the limitations of GENIE?
The GENIE design is intended to extensible. Limitations of version 2 include:

• Unlike BeoPEST, which includes MPI and TCP/IP communication, GENIE version 2 supports only TCP/IP communication.
• Only IP v4 is fully tested; IP v6 is available in version 2 but only cursorily tested.

Common Issues with GENIE
Will GENIE remotely copy and launch slaves on distributed resources?
No, GENIE manages the communication between a non-parallel computing code and the distributed computing resources. The user must manually copy the run directory and launch GENIE. The ability to remotely copy the run directory and launch GENIE is provided by companion software PESTCommander (Karanovic and others, 2012).

Why doesn't the GENIE Master see the GENIE Executer?
GENIE communications are handled through the widely used TCP/IP protocol that is standard for Internet communications. Firewall and computer security policies commonly close TCP/IP ports. Although GENIE can use any of the standard TCP/IP ports (ports between 1024 and 65535), the port chosen by the user will need to be unblocked by the wide-area/local area networks, and the Windows operating systems running GENIE components. Moreover, the same port number needs to be input to both the GENIE master and associated GENIE slaves.

Why are there a minimum of 3 DOS command windows needed to run GENIE rather than 2 DOS command windows needed by BeoPEST and PPEST?
Because GENIE is a general run manager that can be applied to a variety of non-parallel computing codes; one DOS command window is needed to run the original non-parallel computing code (the outside or calling program), one is needed to run the GENIE in Master mode, and one or more are needed to run the GENIE in executer (or client) mode. PPEST and BeoPEST are specialized and non-general parallel computing codes, thus have GENIE Master-type capabilities integrated within the original code.

Non-USGS GENIE Reports, Articles, and Related Background Material

(See Publications tab above for official USGS publications)

Dahlstrom, D.J., and Carter, J.T.V., 2013, Inverse Modeling with PEST++ and GENIE. Groundwater 51(2): 162-167. doi:10.1111/gwat.12021

Doherty, J., 2014a, PEST, Model-independent parameter estimation-user manual (5th ed., with slight additions): Brisbane, Australia, Watermark Numerical Computing.

Doherty, J., 2014b, Addendum to the PEST manual: Brisbane, Australia, Watermark Numerical Computing.

Doherty, J., 2015, PEST - The Book: Calibration and Uncertainty Analysis for Complex Environmental Models. Watermark Numerical Computing, Brisbane, Australia, ISBN: 978-0-9943786-0-6, 227 p.

Muffels, C.T., Schreüder, W.A., Doherty, J., Karanovic, M., Tonkin, M.J., Hunt, R.J., and Welter, D.E., 2011, A model independent TCP/IP run manager, in MODFLOW and More 2011: Integrated Hydrologic Modeling, Proceedings of the 10th International Conference of the International Ground Water Modeling Center. Golden, CO: Colorado School of Mines.

Schreüder, W.A., Muffels, C.T., Tonkin, M.J., Doherty, J., Hunt, R.J., and Welter, D.E., 2011, Efficient use of parallel resources using PEST, in MODFLOW and More 2011: Integrated Hydrologic Modeling, Proceedings of the 10th International Conference of the International Ground Water Modeling Center. Golden, CO: Colorado School of Mines.