Welcome to the GDXRRW home page! GDXRRW is an interface between GAMS and R. It includes functions to transfer data between GDX and R and a function to call gams from within R.

From this site you can download versions of the GDXRRW package for all supported architectures. I also expect to collect some of the common questions and issues about GDXRRW and put them into a FAQ available here.

We would love to get feedback on GDXRRW. Please direct your questions, comments, and suggestions to R@gams.com.

Latest stable version

• GDXRRW 1.0.4, 19 Feb 2018: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

Builds with R 3.5.1 (rename files after downloading to lose the _r351

• GDXRRW 1.0.4, 10 Aug 2018 (Built with R 3.5.x): [ OS X 64-bit binary] [Windows 32- and 64-bit binary]

Latest trunk (development) version

Older versions

• GDXRRW 1.0.2, 19 Oct 2016: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.5.4, 20 Apr 2015: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.5.1, 28 Jan 2015: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.2.0, 18 Oct 2012: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.0-4, 2 Jul 2012: [source package] [Linux 64-bit binary] [OS X 64-bit binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.0-3, 2 Nov 2011: [Linux 64-bit binary] [OS X fat binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.0-2, 16 Aug 2011: [Linux 64-bit binary] [OS X fat binary] [Windows 32- and 64-bit binary]

• GDXRRW 0.0-1, 2 Aug 2011 (Initial alpha release): [Linux 64-bit binary] [OS X fat binary] [Windows 32- and 64-bit binary]

The GDXRRW package depends on the reshape2 package. Before installing GDXRRW you should install the reshape2 package, e.g. by running install.packages(“reshape2”)

installing from source: In the R environment, navigate to the directory containing the source package file (see download links above), and, substituting the version number for a.b.c, type: install.packages(“gdxrrw_a.b.c.tar.gz”) or, depending on the version of R, install.packages(“gdxrrw_a.b.c.tar.gz”,repos=NULL,type=“source”)

Linux installation: In the R environment, navigate to the directory containing the binary package file (see download links above), and, substituting the version number for a.b.c and the appropriate platform info, type: install.packages(“gdxrrw_a.b.c_R_platform-info.tar.gz”) or, depending on the version of R, install.packages(“gdxrrw_a.b.c_R_platform-info.tar.gz”,repos=NULL)

OS X installation: In the R environment, navigate to the directory containing the binary package file (see download links above), and, substituting the version number for a.b.c, type: install.packages(“gdxrrw_a.b.c.tgz”) or, depending on the version of R, install.packages(“gdxrrw_a.b.c.tgz”,repos=NULL)

Windows installation: In the R environment, navigate to the directory containing the binary package file (see download links above), and, substituting the version number for a.b.c, type: install.packages(“gdxrrw_a.b.c.zip”) or, depending on the version of R, install.packages(“gdxrrw_a.b.c.zip”,repos=NULL)

If you are relatively new to R you may not have installed R packages before. More complete documentation on installing and managing packages is available in the R manual, but you can also follow these simple steps for getting set up to install packages in user space.

1. Find out the name of the user-specific package installation directory: this is given by the command Sys.getenv(“R_LIBS_USER”)
2. Verify that this directory exists. If it does not, then create it and restart R.
3. Verify that this directory is now the first directory in the library installation/search path: do .libPaths()
4. Do the installation, e.g. using one of the install.packages commands above

The Safari browser un-gzips the .tgz files R uses for packages on the Mac. There are several ways to work around this issue: you can change this behavior in Safari by changing your preferences (as described here), use a different browser like Chrome or Firefox, use the .tgz files included in the GAMS distribution, or use a command line tool like curl to download the .tgz files.

In some cases a source package install will fail when using a tar.gz file as input, with a message like Error in rawToChar(block[seq_len(ns)]). This has been traced to a problem with the untar() function in some R distributions. As a workaround, you can manually un-tar and re-tar the source package, e.g.:

1. mv gdxrrw_0.5.4.tar.gz foo.tar.gz
2. gunzip foo.tar.gz
3. tar xf foo.tar
4. tar czf gdxrrw_0.5.4.tar.gz gdxrrw

After you install GDXRRW you may want to verify that the installation is functioning correctly. The GDXRRW package is not loaded by default, so before the package is available you must load it. This is easily done with the library(gdxrrw) command. Once the package is loaded, you can get help on the package as a whole (help(gdxrrw)) or the functions or data elements in the package (e.g. help(igdx)).

For GDXRRW to work it must load shared libraries from the GAMS system directory. One common problem is a failure to find the GAMS system directory and these shared libraries. The igdx command can be used to show what GAMS system directory GDXRRW has found (igdx()) and to point GDXRRW to a particular GAMS system directory if so desired (igdx(“/usr/local/gams/24.4.1”)). Depending on your system setup, environment variables, and how you start R, GDXRRW may not need any help finding the GAMS system directory, but the igdx command allows this binding to be queried and customized if desired.

The R sessionInfo() command tells you the version number of packages you have loaded, including gdxrrw. If you need SVN revision info, some commands print that, e.g. rgdx(“?”).

The help pages for the other GDXRRW functions include examples that you can run to quickly verify your system is working.

If you are seeing GDXRRW for the first time you should look at the examples and tests that come with GDXRRW. For example, each of the functions wgdx, rgdx, igdx, and gams comes with integrated help and examples that can be accessed in the usual way within the R environment, e.g.: help(rgdx). In addition, more complete tests and more substantive examples are included in the gdxrrw package. For more recent package versions (0.6.0 and following), package tests are collected in the tests directory. Earlier versions put the tests in extdata. The sample commands below assume a recent version and sufficient access permission in the package install directory. If you lack such permission copy the files before using them. Some helpful R commands for testing are:

• setwd (paste0(path.package('gdxrrw'),'/tests'))
• execute the tAll.R script (source(“tAll.R”) to run the tests

Some examples are collected in the extdata directory:

• setwd (paste0(path.package('gdxrrw'),'/extdata'))
• In subdir TSP, execute the egTSP.R script to export data for a traveling salesman problem to GAMS, solve it, and return the solution to R for plotting
• In subdir LS, run the egLeastSquares.gms model from GAMS - this calls R as part of the GAMS run to do the data fitting and return the results

R provides multiple ways to customize the environment at startup, e.g. to start with the GDXRRW package loaded. I prefer to use the .Rprofile script that is executed from the HOME directory when R starts up.

• For a description of how R starts up: help(Startup)
• To find your home directory, especially useful on Windows: Sys.glob(“~”)
• To load non-default libraries like GDXRRW on Linux startup, use a .Rprofile with this line:

    .First <- function() { library(gdxrrw) ; igdx("/home/sdirkse/gams/24.4.1")}
    

• On 64-bit Windows you can run 64- and 32-bit R. The .Rprofile handles the initialization easily:

    .First <- function() {
      library(gdxrrw)
      a <- R.Version()$arch
      if (a == "x86_64")  igdx("/gams_64/24.4.1")
      else                igdx("/gams_32/24.4.1")
    }
    

• To get system information, try Sys.getenv:
  • Sys.getenv(“R_LIBS_USER”) helps to find user-installed packages
  • Sys.getenv(“R_USER”) helps to locate the .Rprofile file, especially on Windows
• To get version information: R.Version

• Initial alpha release

• fix the double usage of the file “eurodist.gdx” in extdata.
• optional arg “compress=FALSE” to rgdx.param and rgdx.set so factors are compressed instead of having levels running over the whole range of UELs
• optional arg “ts=FALSE” to all rgdx wrappers to get the explanatory text returned as an attribute
• rgdx.param: change default name of value column from val to value
• gams() crashed or failed if no data is passed back
• remove the crosstab option to rgdx.param: better to use the recast function from the reshape package
• rgdx returns the symbol name using the case from GDX, not the case as specified in the search
• add wrappers for writing data to GDX: wgdx.df, wgdx.scalar, and wgdx.lst

• fix gams(x) so x is like the command line, not just the GAMS source file name
• remove output=std global control for form of output
• add a flag similar to the GDXXRW squeeze = yes | no flag for handling zeros
  1. when reading GDX, default squeeze = TRUE/nonzero/'yes' squeezes zero and EPS out of the data returned to R, while squeeze = FALSE/0/'no' returns the zeros and EPS values. This only matters when the data is returned in a sparse format.
  2. when writing GDX, default squeeze = 'yes' doesn't store zeros in GDX. With squeeze = 'no' zero values are stored explicitly in GDX. With squeeze = 'eps' zero values are stored as EPS in GDX.
• fix and update the treatment of GAMS special values (NA, UNDF, +/- infinity, EPS) and zero.
• build and install with a NAMESPACE - required with the new R 2.14 and supported previous to that.

• fix unsqueezed read of variable/equation
• fix issue with backward compatibility and GLIBC on Linux
• add capability to return symbol info from a GDX file: gdxInfo (gdxName,dump=TRUE,returnList=FALSE,returnDF=FALSE)
• fix rgdx.scalar for zero scalars
• fix NULL returns (make them invisible), some other returns now invisible also
• add an experimental wgdx.reshape wrapper to facilitate reshaping/preparing dataframes for writing to GDX. This wrapper is subject to change, and comments are welcomed. A vignette is available [here].
• add silent=FALSE arg to igdx

• internal code reorganized
• several fixes for memory allocation bugs: thanks gctorture!
• tests expanded and reorganized
• fix bugs when te=TRUE (i.e. when we read text elements with sets)
• set dimnames for output $val and $te when form=full: allows access to data directly via UEL labels.
• dramatic speedup for common use cases when using user-provided UEL filters
• use the domain information in the GDX file as the default domain
• better support for set aliases
• fix to domain information in gdxInfo
• add new arg useDomInfo=TRUE to rgdx family
• add $domains to rgdx output: returns domain info from GDX file if available

• implement writing of variables and equations. In the process, re-implement/fix/test the variable and equation reading and writing for all field specs.
• implement field='all' for variable and equation reading/writing.
• fix mixup of .lo/.up fields when reading variables or equations (r36158)
• fix problem when reading zero scalars (r36159)
• fix filtered scalar read: corner case (r36163)
• fix gams() call so it uses the GAMS sysdir (the GDX load path, really) as set or shown by igdx() (r40555)
• update igdx() to allow return of GDX load path (r40607)
• update wgdx() to accept any and all outputs from rgdx() - ignoring some unexpected fields, etc. (r40615)
• various fixes to reduced time and memory requirements, fix memory leaks, improve error messages, etc.
• expand wgdx.lst functionality to accept multiple arguments with symbol data. Each arg contains data for a single symbol or is a list of such args. Drop wgdx.df and wgdx.scalar at the same time, subsumed by wgdx.lst.
• document allowed requestList elements (r41485)
• add domain symbol names to dimnames of return$val when form='full' (r41519)
• add code to read and write aliases as aliases. Previous versions of rgdx() returned the data for set K when reading aliases for K. A new arg to rgdx, followAlias=TRUE, controls this behavior. (r41571,r41584,41599)
• write relaxed domain info to GDX when available (i.e. when specified by the user) (r41596)
• write relaxed domain info also when writing a data-frame (r41938)
• check for NA when reading factors as sets, abort if found (r43965)
• fix reading sets or parameters as dataframes: subtle bug when card(implied_index_set) >= 1e+5 (r43981)

• query option gdx.inventSetText and use to control set text when GDX contains none (r44385)
• add check.names arg to rgdx.param, rgdx.set: like check.names in data.frame (r44399)
• add te=FALSE arg to rgdx.set (r44415)
• query option gdx.inventSetText and use to control behavior when writing empty string for set text (r44571)
• fix bug in wgdx.reshape when also reordering index columns where the time column is not rightmost (r46249)
• enable set text when writing dataframes to GDX (r50551)
• fix handling for R NaN values (r50627)
• move tests from extdata to tests (r50629)
• add domInfo field or attribute containing meta-information about the domain, e.g. domInfo=“full” when reading a GDX containing full domain info. Other possible values include “none”, “relaxed”, “compressed”, and “filtered. As a consequence, we don't return domain names of “_compressed” and “_user” any more. Instead, we return the best domain names we have, and the user can check domInfo to see how what is returned relates to what is in the GDX file. Thanks Wietse! (r51734,r51737,r51832)
• add domInfo to data frame returned by gdxInfo, and add some missing text and domain columns in this dataframe (r51760)

• various config, doc, and source cleanups to try make CRAN happy
• updates to way names are handled (data frame column names == indexing set names) (r51955)
• adjust to handle dplyr tbl format as input (r58974)

• fix problem loading a compressed GDX file (gdxSetLoadPath was called incorrectly)

• handle empty factors when checking data frames (r65528)
• fix errors when reading empty data into a dataframe with compressed=T (r65529)

• still at 1.0.4

• Possible change: remove or deprecate requestList arg on rgdx, replace with separate named args.