NEWS/Changelog for Rserve
---------------------------
NOTE: Rserve 1.8 series has many new features, but it has also removed
support for control commands. If you absolutely need them,
please use Rserve 1.7 series and notify me so I can assess
whether they may be resurrected for the 1.9 series. If you don't
know what it means, just ignore this note.
1.8-15
o The static HTTP/HTTPS handers are now officially supported
so Rserve.http.add.static() is now exported and documented.
Also Rserve.http.rm.all.statics() has been added to allow the
removal of all handlers.
o There were several bugfixes in the handling of paths in the
static http handlers (sanitization was done on the full path
instead of the request part only and separating / are now
inserted where necessary).
o Shorthand arguments --RS-simple-http <path> and
--RS-simple-http-only <path> have been added to the command
line Rserve. They disable QAP and register a static http
handler using the supplied <path> as document root. If
http.port has not been set, it will default to 8080.
Note that the <path> should be absolute since Rserve changes
the working directory to / when daemonizing (or use
--RS-set daemon=no). The --RS-simple-http variant continues
to process other handlers (static or R) while the
--RS-simple-http-only variant will respond with 404 if
the resource is not found (see Rserve.http.add.static()
documentation).
QAP can be re-enabled if desired by subsequent arguments. If
the desire is to only run a HTTPS server then --RS-set http.port=0
can be used to disable the default and then add all necessary
arguments (or a config file) for the TLS part.
o added run.Rserve(background=TRUE) mode which meshes server
processing loop (i.e., the loop which accepts new connections)
and the R session event loop. The function returns as soon
as all the server ports are bound (essentailly immediately).
This allows the server R process to run R code in the main
session after the server was started and thus influence future
client connections. This is currently only supported on unix.
Please note that any changes to the server R session only
affect new connections, not existing ones.
If a function .Rserve.served is defined in the global
environment then it will be called after each new accepted
client connection in the main R session.
Function stop.Rserve() can be used to stop and remove all
servers started with run.Rserve(). Note that all existing
client connections will remain intact.
NB: This functionality is currently considered experimental
and the return value is undefined (currently TRUE on success,
but may return other information in the future to enable more
fine-grained control over the background servers).
1.8-14
o Windows: use pkg-config if available (many thanks to Tomas
Kalibera for the work on the toolchain!)
o Close listening socket when re-attaching session. (#155)
o Fixes to the sample C++ client.
1.8-13 2023-11-28
o replace sprintf with snprintf even in safe cases to avoid
compiler warnings.
1.8-12 2023-08-17
o added a work-around for buggy libcrypt (first seen in Ubuntu
22.04) which incorrectly uses salt leading to authentication
error if crypt is used even with a correct password. (#192)
o parsing of dates in conditional requests could be off, leading
to fewer 304 Not Modified responses than expected (proxy).
o added experimental built-in feature in Rserve's HTTP(s) server
to serve static files directly (similar to the feature of the
forward proxy). Multiple prefixes can be registered to serve
static content from different paths using the
Rserve:::Rserve.http.add.static() function. Static handlers
are processed before R handers, but they can pass on requests
if the file is not found. Conditional requests are supported.
1.8-11 2022-11-28
o fix possible buffer overflow attacks in WebSockets code (many
thanks to Dane Henshall for performing a security audit!)
o clean up code to avoid mixing 32-bit and 64-bit integers on
64-bit platforms
o enhance `Rserve.eval()` to report the condition object for
error conditions in the `condition` component of the
`Rserve-eval-error` object (#154) and to support custom
calling handlers.
This is done by adding a new argument `handlers` to
`Rserve.eval()` which allows registration of calling handlers
for the duration of the evaluation. The default
is to register an `error` handler which stores the condition
(via `Rserve_set_last_condition`) so `Rserve.eval` can pick
it up and return in the `condition` component. To revert
to previous behavior, use `handers=NULL` which prevents the
registration of handlers.
o Issue a warning if eval/source configuration directive is used
in `run.Rserve()` since they only affect stand-alone Rserve
daemon startup and thus setting them in an R session is likely
a mistake (as the user can run R code directly in the session
prior to calling `run.Rserve()` instead).
1.8-10 2021-11-25
o reduce the size of the authkey (nonce) in CMD_reqKey to
256 bytes such that the payload of CMD_secLogin can fit
into one encryption block (for 4096-bit RSA key). This
improves the nonce entropy size from 50 bytes to 256.
Note that effectively the maximum safe size of the
authentication payload is 206 bytes. Larger sizes can be
transmitted, but should be considered unsafe due to
the second block not including any nonce.
o Prevent replay attacks on non-forked servers using
CMD_secLogin without CMD_keyReq. (Thanks to
Dane Henshall for reporting!)
Deployments relying on CMD_secLogin for authentication
should upgrade to this version to avoid vulnerability.
Note that clients do not have to be updated.
o Respect CPPFLAGS and LDFLAGS from R CMD config
1.8-9 2021-11-05
o move m4 under tools according to R-exts 1.2
o several fixes to QAP encoding/decoding, use R_xlen_t
type for indexing if available (previously Windows
builds used 32-bit index due to long int size limits).
o protection fixes for serialized commands
o remove rsio entirely since it is not used
1.8-8 2021-10-31
o add basic ulog logging for QAP1 even outside of OCAP mode (#169)
o TLS configuration options now check the success of the
operation and will issue a warning if it failed. Most
importantly `tls.key` must succeed in order for the
server to support TLS.
o New option `tls.client` governs the treatment of client
certificates. There are following possible values:
none - does not ask nor check the client certificate
request - request certificate from the client, but
do not check its validity
require - require client certificate, if not present
or not valid, reject the connection
match:... - client's common name must match exactly one
of the listed (comma-separated) names. No spaces
are allowed around the commas.
prefix:... - the client's common name must start with the
specified prefix
suffix:... - the client's common name must end with the
specified suffix
The options `match`, `prefix` and `suffix` imply `require`.
Each of them can be specified only once. However, it is
possible to use up to one each in which case the requirement
is treated as logical "or", e.g.,
tls.client match:foo.bar
tls.client suffix:me.com
will allow both clients `foo.bar` and `foo.me.com`.
NOTE: `none`/`request`/`require` are mutually exclusive. If
more than one is specified, the behavior is undefined
(currently the last one wins but that shold not be relied
upon). All comparisons are case-senstitive (sorry, no regex
support at this point, because we do not want to depend on
PCRE just yet).
If the `tls.client` option is not specified, then the default
behavior is one of `none` or `request`. In that case no attempt
is made to control the TLS behavior so it depends on the SSL
library's default whether it asks for the client certificate
or not, but in either case it is not checked.
NOTE: probably the best way to use the new feature is to use
certificates issued by your organization with
`tls.client require` and `tls.ca` set to the organization's CA
certificate which means you don't need to check client names
and yet no unauthorized client can connect.
o -DULOG_STDERR now correctly produces ulog output on stderr
and ulog output is visible in the debug build
o TLS shutdowns are performed explicitly and sockets closed
(only affects non-forking servers)
o several possible leaks of arguments under error conditions
were fixed
1.8-7 2020-12-17
o proxy: fail with an error if no servers can bind (#152)
o proxy: fail if SSL cannot be initialized
o include err.h SSL header outside of debug mode
1.8-6 2020-06-11
o proxy: handle WebSockets control frames (PING/PONG/CLOSE)
proper.
o proxy: fail with an error if key/certificate cannot be
loaded at start time. Both -k and -c are required to
enable TLS/SSL. Also the proxy will issues an error if -k
is used and SSL support has not been available at compile
time. (#127)
o proxy: linking no longer hard-codes `-lssl` (it effectively
failed compilation when SSL was not available).
o proxy: added -S <host>[:<port>] which allows automatic
registration and de-registation with a server manager
when the service is available (e.g., this can be used
to keep the compute node list up-to-date automatically
when load-balancing via the nginx reverse proxy).
o support large vectors in QAP encoding/decoding
o restore some Windows compatibility - at least to the point
where it compiles. It is still strongly discouraged to use
Windows given how limited the OS is and how it prevents
any sensible use (Windows only supports single-client,
single-thread, cooperative mode - so for toy uses only).
o content-type header entry is lower-cased only up to a
semicolon since trailing options may be case-sensitive. (#149)
1.8-5 2017-03-25
o add support for callback in OCAP idle mode. If the function
.ocap.idle() exists in the global environment and the
configuration option
use.idle.callback yes
is set then that function is called inside each OCAP iteration
after a timeout of 200ms (i.e., if R is idle in the OCAP loop
with no input from any side for at least 200ms).
o control commands are now officially disabled since they never
worked reliably in the 1.8 series after the the rsio re-write.
1.8-4 2015-12-23
o bugfix: when using compute separation on some Linux systems the
death of the compute process may remain undetected. Rserve
now actively detects if the compute process has terminated and
will signal accordingly. The same goes of a compute process
whose parent has died.
o bugfix: make sure headers are correctly terminated when using
forward proxy and R scripts.
o bugfix: static files served by the proxy had duplicate (and
possibly conflicting) content-type headers.
1.8-3 2015-07-20
o add support for the notion of a context object which can be
retrieved/set using Rserve.context() function. If
io.use.context yes
configuration option is set then the context is passed
as-is in OOB console callbacks as the first argument after
the callback name.
Rserve.eval() can also safely set a context object for the
duration of the evaluation which guarantees that any
previous context is restored upon exit - both on error or
success.
o added WebSockets/QAP proxy and HTTP server. It is a
stand-alone program that can be used to separate the R process
from the network to enhance security. typical setup is for the
proxy to listen on an outside port and connect to Rserve on a
local (unix) socket which is restricted to the user running
the proxy.
It is enabled by default and installed into the package's libs
directory along with the Rserve binaries. It is, however, not
installed into $R_HOME/bin. The building of the proxy can be
disabled via --disable-proxy configure directive.
1.8-2 2015-05-20
o fix bug in the WebSockets protocol implementation where frames
over 64kB from the client may be parsed incorrectly.
1.8-1 2015-04-08
o add ulog() R function to allow custom logging from R code
o add "fork here" directive which allows to spawn multiple
pre-forked instances of Rserve which all share the same
memory. The rest of the configuration file is then ignored in
the forked instance while the parent server instance continues
to read on. This can be useful on machines with many cores and
very short requests where the serial fork() time for each
connection is in the order of the computing time. Note that by
design this only works in configuration files and cannot be
used on the command line.
o daemonize only *after* all servers have been bound such the
exit code can be used to detect if Rserve has actually
started successfully
o add Rserve.eval() function to perfrom REPL-like evaluation
o add a new config option to close all stdio file descriptors
(stdin, stdout, stderr) on daemonization:
close.all.stdio enable
For compatibiliy the default is false.
o Rserve will honor --silent command line option (destined for R)
to not print the "Rserve started .." line.
o add support for console input OOB message, enabled via
console.input enable
When enabled R will request input to ReadConsole by sending a
OOB message to the client, expecting back a string to be used
as input to the console. It is only honored if console.oob is
also enabled.
1.8-0 2014-09-03
o new logging mechanism has been added to Rserve. It can
be enabled using the
ulog <path>
directive and uses syslogd-compatible unix socket or UDP
datagram transfer to log events. Currently, only OCAP
events are logged.
o OCAPs can now be tagged with a name
o avoid duplicate symbols by not loading the Rserve dylib
inside an embedded Rserve instance. This avoid issues
of calling the wrong symbols, but it implies that
run.Rserve() can no longer be used from within a
child proces of an embedded Rserve instance (which
you really don't want to do anyway).
o symbols are now registered *before* processing source/eval
directives from the configuration file
o added new option to remove the working directory on
session close recursively - activated by
workdir.clean enable
If not set, the default is to only remove it if it is empty
(on the assumption there are files that will be served beyond
the life of the session).
o added workdir.parent.mode option which specifies the
permissions on the parent directory of session working
directories. The default is 0755. Unlike workdir.mode this one
does not respect the umask (beause it may be shared and thus
may need to include permissions that are otherwise not
granted).
o the data offset field in the QAP header is deprecated and can
be used for message ID if enabled using
msg.id enable
In that mode all responses will copy the message ID from the
request and new requests (such as OOB send/msg) will use
randomly generated message IDs. This allows clients to more
easily track responses for nested commands. Note that this is
backwards-compatible if old clients use the convention of 0
data offset which was the norm (as long as OOB is not used).
o nested evaluation is supported in OCAP mode. This means that
OCcall followed by another OCcall can be processed before the
first call terminates if the R side issues an OOB message in
which case the second OCcall will be nested within the first
call.
o allow expansion of environment variables in the configuration
file, i.e., ${VAR} will be replaced with the contents of the
VAR environemnt variable. This only applies to the right-hand
side of configuration directives.
o added command line argument --RS-set which allows any
configuration directive that is accepted in the configuration
file to be specified on the command line as
--RS-set directive=value
o fix a typo in the "daemon" configuration option which was
looking for "deamon" instead.
o added support for console callbacks. They can be enabled using
console.oob enable
directive and they result in OOB messages being sent on
console I/O operations.
o add a configuration option "shutdown disable" that will
disable the shutdown command. This makes is a bit harder
for unprivileged users to shut down the Rserve instance.
It does not disable the ctrlShutdown command, so users that
have control command access can still issue a shutdown.
Note that this option will eventually become the default
so consider specifically using "shutdown enable" where
CMD_shutdown is still needed.
Note that the shutdown command is not functional in cases
where user switching is performed anyway, so is it only used
for backward compatibility. Only ctrlShutdown is guaranteed
to work.
o add a configuration option "oob.idle.interval" which will
trigger automated OOB send of "idle" command after a
specified interval. Currently it only works with
WebSockets.
o add a configuration option "forward.stdio enable" which
will enable capturing of stdout/err and forward it to
the peer via OOB send messages. Note that this requires
thread support and uses additional file descriptors.
It in currently only available in OCAP WebSockets.
o bugfix: pairlists were incorrecty protected when parsing
SEXPs (qap_decode) in messages from client to server (#16).
1.7-3 2013-08-21
o the handling of server configuration modes has been
inconsistent for some combinations (mostly affected were
combinations involving WebSockets upgrade and OC mode on
HTTP/WS servers). Also the websockets.qap.oc configuration
option has been misspelled.
o HTTPS->WSS upgrade is now supported
o mkdist -i installs the built package, use -c for check
o Windows compatibility has been restored thanks to
David Champagne from Revolution Analytics
1.7-2 2013-08-12
o when uid/gid is changed, create a new tempdir() and set its
permissions as well as the wokring directory's owner to match.
o bugfix: if the first command is not any of the eval family
the Rserve may respond with an additional, spurious error
response
o deamonized server would record incorrect pid when pid.file
is used (i#5). The pid is now removed on clean sutdown.
o added support for keep.alive configuration option - it is
global to all servers and if enabled the client sockets are
instructed to keep the connection alive by periodic messages.
1.7-1 2013-07-02
o remove a spurious character that prevented compilation on Suns
o add OPENSSL_INCLUDES precious variable that can be used to
point to non-standard location of OpenSSL headers
o check the usability of OpenSSL headers before enabling TLS
support
o added the choice of GPLv2 or GPLv2 with OpenSSL linking exception
1.7-0
*** ---- HEADLINE NEWS ----
*** new protocols: HTTP, HTTPS, WebSockets and TLS/QAP
*** added protocol switching (from QAP to TLS/QAP) via CMD_switch
*** The R client was moved to RSclient package
*** New in-session server support with run.Rserve()
*** Out-of-band messages via self.oobSend()
*** user-based uid/gid switch, MD5/SHA1 stored passwords
*** preliminary IPv6 support; RSA secure authentication
*** .Rserve.done global env hook for optiopnal cleanup
*** auth.function setting to allow custom authentication
*** Object-capability mode for hardened, secure services
*** ---- END - see below for details ----
o This is the first release in the new 1.x Rserve series. Many
of the Rserve internals have been re-written or cleaned
up. The original protocol remains the same (so all clients
that worked with Rserve 0.6 will continue to work), but the
suite of available protocols has been extended (see below).
o added support for multiple protocols in Rserve:
** QAP **
this is the original Rserve protocol used in Rserve 0.x
series. It works over TCP/IP and unix sockets. It is
enabled by default and can be disabled using
"qap disable" configuration directive.
** HTTP **
this is very similar to the built-in R http server except that
on unix it forks on connection so it allows parallel separate,
persistent connections. It requires a worker function
.http.request to be defined in the session which will handle
incoming requests. This allows the use of facilities like
FastRWeb without an external webserver. This protocol is
disabled by default and can be enabled by setting "http.port"
configuration directive to the desired port to listen to.
(Also see TLS support below for https server).
The http.raw.body configuration option (default is false which
results in the same behavior as Rhttpd) can be used to force
passing body in raw form to the handler (useful for the
FastRWeb handler which does its own parsing).
** WebSockets **
this protocol is used by HTML5 web browsers for direct access
to R with a persistent connection. This allows implementation
of websites that have a dedicated R session (and could be used
as an R console). There are two subprotocols supported by
Rserve:
-- WebSocket(*, "QAP") --
this is a tunnel for the original QAP protocol through
WebSockets. It requires a browser capable of binary
WebSockets protocol (version 01 or higher). It allows very
efficient data transfer, typically by loading ArrayBuffers
directly into GPU or CPU.
It is disabled by default and can be enabled using
"websockets.qap enable" configuration directive.
(If no subprotocol is specified, QAP is assumed)
-- WebSocket(*, "text") --
this is a very simplistic protocol using plain text frames
to send input to R and output from R directly as text. It
acts essentially as an R console. This protocol works with
any WebSockets implementation including version hybi-00 and
hixie-75/76
It is disabled by default and can be enabled using
"websockets.text enable" configuration directive.
NOTE: The textual WebSockets protocol does NOT provide any
authentication mechanism, so use with extreme care as you
are essentially giving any user with a web browser access to
R and thus to the shell.
In addition to enabling each or both subprotocols, the port on
which the WebSockets server should listen must be specified in
the configuration using "websockets.port" directive, for
example "websockets.port 8080". Alternatively, the HTTP server
can be enabled to allow connection upgrade to WebSockets on
the same port with "http.upgrade.websockets enable"
NOTE: remember that the default in Rserve is to disallow
remote connections, so you may need to use "remote enable" in
order to use WebSockets or HTTP in practice, since the point
is to serve remote machines. Typically, if both QAP and either
HTTP or WebSockets are used, it is recommended to use QAP on a
local unix socket for better access control.
o Rserve now supports SSL/TLS connections on QAP, HTTP and WS
protocols. The TLS key/CA entries are common for all
protocols. The relevant new configuration directives are:
tls.key <private key PEM file>
tls.cert <server certificate PEM file>
tls.ca <CA PEM file issuing the above certificate>
SSL/TLS can be used in several ways: with separate port for
TLS connections or by switching protocol form a regular QAP
connection using CMD_switch with "TLS" as argument. The latter
can be enabled using
switch.qap.tls enable
Enabled switching is advertized by Rserve with the presence
of a "TLS" entry in the ID string. Keys and other TLS entries
must be initialized in order for TLS to be enabled.
Dedicated TLS servers can be enabled by specifying the port
for the corresponding protocol:
qap.tls.port <port> - for Rserve/QAP
http.tls.port <port> - for HTTPS
websockets.tls.port <port> - for WebSockets
(there are synonyms "https.port" for "http.tls.port" and
"tls.port" for "qap.tls.port", however, the *.tls.port
versions are preferred for clarity)
The use of TLS protocols is encouraged where sensitive data
is transmitted or when requiring secure authentication. For
QAP protocol, using TCL/QAP with SHA1 passwords (also new, see
below) is the currently recommended way where authorization is
required. The only drawback is increased CPU utilization
during transfers casued by the encryption and the fact that
TLS-enabled clients must be used. See also RSA secure
authentication (CMD_keyReq + CMD_secLogin) for a different
method if only the authentication step is to be encrypted.
When enabling TLS tls.key and tls.cert are mandatory, tls.ca
is optional to establish CA chain of trust (whether this is
needed depends on the client and the certificate). To generate
a key and self-signed certificate, you can use something like
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 365 -in server.csr \
-signkey server.key -out server.crt
NOTE: TLS services are started **in addition** to any other
servers, i.e., if you want to enable TLS/QAP only, you have to
set tls.qap.port but also add "qap disable" to disable the
plain Rserve access.
o QAP servers (classic Rserve QAP, QAP/TLS and WebSocket/QAP)
support object-capability (OC) mode in which all Rserve
commands are disabled except for CMD_OCcall. All messages
including the initial handshake are always QAP and the initial
message defines capabilities (here opaque references to
closures) that can be called. This mode can be enabled using
qap.oc enable ## for Rserve QAP and QAP/TLS
websockets.qap.oc enable ## for WebSockets/QAP
In this mode the configuration *must* define a function
oc.init (typically using eval or source configuration
directives) which has to supply OC references that can be used
in calls. If the evaluation of oc.init() fails, the connection
is closed immediately. The use of invalid OC references or any
other commands other than CMD_OCcall results in immediate
connection termination. This allows creation of hardened,
secure services that can disallow arbitrary code execution.
NOTE: this mode is inherenty incompatible with all classic
Rserve clients. The first four bytes of the initial packet are
"RsOC" instead of "Rsrv"
o Rserve can now be started from within an existing R session
using run.Rserve() command. This allows the user to prepare a
session "by hand", run Rserve from within that session and go
back to the session (by shutting down the server or sending an
interrupt). This allows the use of Rserve even without the
presence of libR.
o Rserve now supports out-of-band (OOB) messages. Those can be
sent using the self.oobSend() [one-way] and self.oobMessage()
[roundtrip] functions from within code that is evaluated in
Rserve child instances. OOB messages are not used by Rserve
itself but offer asynchronous notification to clients that
support it (one typical use are WS-QAP1 tunnels to web
browsers that allow status updates as R code is evaluated).
o Rserve accepts additional command line arguments:
--RS-source <file> (same as "source <file>" in cfg file)
--RS-enable-remote (same as "remote enable" in cfg file)
--RS-enable-control (same as "control enable" in cfg file)
o The Rserve package no longer includes the R client. It has
been moved to a separate package "RSclient" so that it can be
used on machines separate from the server.
o There was a bug in QAP storage estimation affecting pairlists,
possibly resulting in buffer overflows. This should be fixed
and an error message will be printed when such overflows are
detected in the future (which hopefully won't happen).
o Bugfix: command line parsing would skip over some arguments
o Passwords file can contain MD5 or SHA1 hash of a password
instead of the plaintext password for non-crypt
authentication. In that case the hash must be lowercase hex
representation with preceding $ sign, so for example user
"foo" with password "bar" would have an entry
foo $62cdb7020ff920e5aa642c3d4066950dd1f01f4d
You can use
echo -n 'password' | openssl sha1
to obtain the SHA1 hash of a password (openssl md5 for MD5
hash - MD5 is probably more common but less secure than
SHA1). This feature makes sure that passwords are not stored
in plain text and thus are safe from local attacks.
o Rserve now has the ability to change uid/gid according to the
user that has been authenticated. The following settings
concern this feature (unix-only):
auto.uid {enable|disable} [disable]
auto.gid {enable}disable} [disable]
default.uid <uid> [none]
default.gid <gid> [none]
The auto.uid/gid directives enable setuid/setgid based on
user's uid/gid. In case no uid/gid is specified with the
username, the default.uid/gid settings will be used. If there
is no uid/gid in the username and no defaults are specified,
the user's authentication will fail. User's uid/gid can be
specified in the passwords file by appending /uid,gid to the
username. If gid is not specified, uid will be used for both
uid and gid. So for example user "foo" (from the above MD5
example) with uid=501 would have an entry on the passwords
file:
foo/501 $37b51d194a7513e45b56f6524f2d51f2
For this to work, Rserve must be started as root. However,
with auto.uid enabled it is safe to do so since Rserve will
prevent any R access until authenticated. You should, however,
use a client capable of secure RSA authentication or use secure
connection such as QAP/TLS as to not send password in
cleartext over the wire.
o Rserve can now be run in a mode where each connection has a
different uid and/or gid such that separate client instances
are isolated. This allows more restricted setup in cases where
instances may not be trusted and need to be sandboxed. The
following configuration directives are associated with this
functionality:
random.uid {enable|disable} [disable]
random.gui {enable|disable} [disable]
random.uid.range {<from>..<to>} [32768..65540]
If random.uid is enabled and random.gid disable then only the
uid of the process is changed. If both are enabled then the
gid is set to match the value of the uid. random.gid cannot be
enabled without random.uid.
To support sandboxing, the permissions on the working
directory can be specified using
workdir.mode <mode>
o If any kind of client-process uid switching is enabled in the
configuration, the permissions on the working directory will
match the uid of the process. Also the working directories are
now named by the process ID to facilitate cleanup.
o Rserve now supports secure authentication even outside of
SSL/TLS. There are two new commands that can be used by the
client:
CMD_keyReq - requests an authentication key from the server
that will be used to perform a secure
login. The kind of the requested key is
specified as a parameter. Currently, only
"rsa-authkey" is supported which returns server
authentication key (authkey) and a RSA public
key which must be used to encode the authkey
and the authentication information (see below).
The RSA key can be compared on the client side
to ensure the authenticity of the server.
CMD_secLogin - secure login. It consists of an encrypted data
stream that will authenticate the user. In the
case of the "rsa-authkey" method, the stream
consists of the authkey and the login +
password, all of which must be encrypted using
server's RSA key.
The RSA key on the server (Rserve) side is specified using
rsa.key <RSA private key file>
configuration file directive. The file is expected to be in
PEM format. You can generate such file, e.g., with:
openssl genrsa -out server.key 4096
where 4096 is the key size in bits. A public key can be
extracted from the private key using
openssl rsa -pubout -in server.key -out server_pub.key
The clients can pre-share the public key by other means to
compare it to the key received from the server as to verify
its authenticity. This will prevent them from sending the
authentication information to rogue servers.
NOTE: if the rsa.key directive is missing, Rserve will
generate a key on the fly when asked for RSA authentication -
although this allows encrypted transmission and thus is safe
from sniffing, it is not safe from man-in-the-middle attacks
where a rogue server intercepts the request and sends its own
public key. Therefore the use of the rsa.key directive is
highly recommended.
The RSA authentication enables the client to a) check the
authenticity of the server (by comparing the RSA public key)
and b) send authentication information encrypted. This method
is highly recommended in cases where a full TLS/SSL encryption
of the entire connection would be too expensive (i.e. in cases
where the data is large and the security of the transported
data is not crucial).
o Rserve has a preliminary IPv6 support. Rserve must be
installed with --enable-ipv6 configure flag to enable it in
the Rserve build. In order to start all servers on IPv6 add
ipv6 enable
to the configuration file. The option is global, i.e. once
enabled it applies to all servers that support it. Note that
not all features work with IPv6 yet - detaching sessions
(they will use IPv4 for re-attach) and remote client filtering
only work with IPv4 at this point.
o Rserve now binds only to the loopback interface in
"remote disable" mode. This is safer and prevents remote DoS
attacks. Previously, Rserve would bind on all interfaces and
check the peer IP address. If desired, you can replicate the
old behavior by adding
remote enable
allow 127.0.0.1
to the configuration (if you don't know the difference then
you don't need this -- if you actually need this, then you
probably want to add more "allow" entries for the machine's
other interfaces as well).
o If a function .Rserve.done() is defined in the global
environment, it will be run after a clean connection
shutdown. This allows custom code to be run when a client
connection is closed.
o If a function .Rserve.served() is defined in the global
environment of the server, it will be run after a client
connection has been served. For forked servers this is just
after the fork(), for co-operative servers this is after the
client conenction has been closed. It is guaranteed that no
other client is served before the call so it can be used to
manage resources that are unsafe to share with forked
processes (e.g. sockets etc.).
o The server and client process can be tagged with extra
information in argv[0] so it is possible to distinguish the
server and children. This behavior can be enabled using
tag.argv enable
Note, however, that this not always possible and it will have
impact on programs that use argv[0] such as killall.
o Added configuration option pid.file and command-line option
--RS-pidfile which instructs Rserve to write its process id
(pid) into that file at startup.
o Added configuration directives http.user, https.user and
websockets.user which take a username and perform
setuid/setgid/initgroups immediately after forking.
This minimizes the amount of code that is run with
elevated privileges in cases where user switching is desired.
o Added configuration directive
daemon disable
which can be used to prevent Rserve from daemonizing. It has
effect only in builds of Rserve that support daemonization.
Note that -DNODAEMON build flag disables daemonization
entirely and can be used in any Rserve version.
o All commands based on eval now also accept DT_SEXP in addition
to DT_STRING. In such case the parse step is skipped and the
expression is evaluated directly. The intended use of this
functionality is to evaluate language constructs and thus
allow calls with both reference and inlined arguments.
o QAP decoding is slightly more efficient and avoids protection
cascades. QAP_decode() has now only one argument and it is
guaranteed to not increase the protection stack when returning
(which implies that it is the responsibility of the caller to
protect the result if needed).
o Both QAP encoding and decoding now use native copy operations
on little-endian machines which can increase the speed
considerably when the compiler cannot do this optimization
on its own (most commoly used compilers don't).
o Assigning logical NAs now uses the proper NA_LOGICAL value
that is also recognized by R. (PR#276)
o Forked child processes will now close all server sockets so
that any server can be restarted without closing existing
children.
o Signal handling has been streamlined: the server process
captures HUP, TERM and INT which will lead to clean
shutdown. Child processes restore signal handlers back to R so
that regular R signal handling rules apply. Note that
interrupt during eval will result in RESP_ERR with code 127
even if try() is used.
--- In order to support new ideas a major re-organization of Rserve ---
--- has been started - almost 10 years after the first release. ---
--- It is time to look ahead again with a new major version. The ---
--- protocol will remain compatible so 1.x series can be used to ---
--- replace the previous 0.x series ---
0.6-8 2012-02-20
o added RSserverEval() and RSserverSource() control commands
in the R client as well as ctrl parameter to RSshutdown().
o added new facility that allows R scripts running in Rserve to
issue control commands if allowed. This feature must be
enabled in the Rserve configuration file using
r-control enable
This will make self.ctrlEval() and self.ctrlSource() functions
available to code that is running within the Rserve
instance. It is also possible to use this feature without
explicitly loading the Rserve package via
.Call("Rserve_ctrlEval", paste(text, collapse='\n'))
.Call("Rserve_ctrlSource", as.character(file))
although this may change in the future.
0.6-7 2012-01-17
o fix processing of login information
**IMPORTANT**: this fixes a serious security hole in the
remote login mechanism! If you rely on authentication,
please make sure you update your Rserve immediately!
(Thanks to Daniel Faber for reporting)
o add a namespace to make R 2.14+ happy
o work around broken readBin() in R 2.14.0 that errors
on unsigned integers (affects R client only)
0.6-6 2011-12-10
o fix a bug that can cause heap corruption due to incorrect
addressing in padding of symbols. Unless extremely long symbol
names are used it is unlikely to have a real effect in
practice, but in theory it could be used to zero targetted
parts of the heap. Thanks to Ralph Heinkel for reporting.
o fix Rserve() call on Windows with quote=FALSE and more than
one argument.
o clarify that sisocks.h is under LGPL 2.1 as well as the other
headers used by clients.
o add support for plain S4 objects (S4SEXP) in assignments
(Note: derived S4 objects - those using other native SEXP type
as a base - cannot be supported properly, becasue there is no
way to distinguish them from S3 objects!)
o Unsupported types in CMD_assign will no longer crash R.
The resulting object is always NULL and an error is printed on
the R side.
0.6-5 2011-06-21
o use new install.libs.R custom installation script in R 2.13.1
to install binaries
o install clients by default on Windows as well
o multi-arch binaries are no longer installed with the arch suffix
in the package root. The canonical place is libs$(R_ARCH) instead.
For now Rserve.exe/Rserve_d.exe are still installed in the root
but they will be also removed in the future as they are not
multi-arch safe.
0.6-4 2011-05-19
o make all buffers capable of using 64-bit sizes. This means
that clients can use more that 4Gb of data on 64-bit platforms
when communicating with Rserve, provided the buffer limits are
either disabled or configured to be high enough. Note that this
does not change the limitations in R with respect to vector
lengths so you still can only use up to 2^31-1 elements.
o bug fix: contrary to the documentation scalar logicals were sent
in the old XT_BOOL format instead of XT_ARRAY_BOOL
o work around several issues introduced in R 2.13.0 for Windows
Rserve() now also allows arguments to be passed to system() for
more fine-grained control of the environment, mostly to work
around bugs and incompatible changes to system() on Windows
in R 2.13.0 (commonly used options are invisible=FALSE to get
back to a more reasonable pre-2.13.0 behavior and wait=TRUE if
using R 2.13.0 that has broken wait=FALSE support).
o In Rserve() startup wrapper, args are now quoted automatically
if quote=TRUE is set. For backward compatilility args are not
quoted by default if they consist of just one string.
0.6-3 2011-01-17
o bug fix: the child process could get stuck in the server loop
after some abnormal return from the child connection code
Thanks to David Richardson for reporting.
o set R_ARCH automatically on Windows if a multi-arch R is
detected (such as CRAN binaries since R 2.12.0)
o add R_ARCH support in Rserve() on Windows to locate the
proper binary
o bug fix: C++ client did not handle new-style lists (introduced
in Rserve 0.5) properly. Thanks to Carl Martin Grewe for
reporting.
0.6-2 2010-09-02
o add support for NAs in character vectors by using a special
"\xff" string. Any string beginning with '\xff' is
prepended by additional '\xff' to remove ambiuguity and clients
should remove leading '\xff' accordingly.
(Note that UTF-8 encoded strings never contain '\xff' so
in most uses it never occurs).
The Java client has been updated accordingly and represents
NA strings with null.
o add a new config file option "interactive" that allows to run
Rserve in interactive or non-interactive mode across platforms.
Previously Windows ran in non-interactive mode and unix in
interactive mode. Non-interactive mode is useful if you want
to prevent R from soliciting user input, but it requires error
option to be set if you don't want to quit R on all errors
(i.e., something like options(error=function() NULL) will do)
Note: on unix the interactivity flag can only be set *after* R
initialization (due to limitation in R) so you still may have
to pass flags like --no-save in order to appease R.
o more Windows fixes - Rserve uses R's own initialization in
recent R versions. This also fixes issues with Win64 and more
recent toolchains.
Note that both Widnows and unix now behave consistently with
respect to interactive mode - the default is now interactive
for both platforms but can be changed in the config file.
0.6-1 2010-05-24
o add a safety margin to the send buffer to avoid crashes when
size estimates are off (e.g., due to re-coding)
o added a very primitive PHP client
o Win64 fixes by Brian Ripley
o added new configuration options:
su {now|server|client} - switches user either immediately
as the config file is loaded ("now", default and always the
behavior of Rserve before 0.6-1), when the server is ready
("server") or when a client is spawned ("client"). The
latter is useful to restrict clients from sending signals
to the server process.
uid, gid config options are interpreted accordingly to
the su value.
cachepwd - {no|yes|indefinitely} - allows Rserve to cache
the password file. "no" = read it at each authorization
(default and behavior before 0.6-1), "yes" = read it when
a client is spawned before su, "indefinitely" = read it
just after the config file (most efficient but changes
are only active after re-start). "yes" has only effect
in unix and can be used to restrict permissions on the
password file such that client code has no access to it
(do does "indefinitely" but can be used anywhere).
0.6-0 2009-10-27
o added support for control commands CMD_ctrlEval,
CMD_ctrlSource and CMD_ctrlShutdown. Those commands provide
control over the server process. The side-efect of eval and
source are then available to all future connections.
Control commands are only available if they are enabled, e.g.,
with the config file entry "control enable". In addition if
authorization is required or the passwords file is set only
designated users will have control access (see next point).
Note that enabling control commands will make Rserve use at
least one file descriptor per active child process, so you may
want to adjust the maximum number of file descriptor in your
system if you expect hundreds of concurrent clients.
o The passwords file format has been enhanced to give
finer-granularity control over the user authorization.
Only users with "@" prefix can issue control commands. The
prefix is not part of the user name for authentication
purposes.
In addition, if the password file contains an entry
starting with "*" it will be interpreted as blank
authorization, i.e. any username/pwd will authenticate. This
may be useful in conjunction with control prefix, e.g., the
following file would give blank authorization to all users but
only the user "joe" will be able to use control commands:
@joe foobar
*
o Windows build cleanup (thanks to Brian Ripley)
o fixed decoding of XT_RAW (it advanced too far), this affected
the use of XT_RAW as non-last element only (thanks to Saptarshi
Guha for reporting)
o don't advertize ARuc if not supported (this bug only affected
systems without crypt support with plaintext enabled and
required authorization)
o add assign support for logical vectors
0.5-3 2009-01-25
o fix SET_VECTOR_ELT/SET_STRING_ELT mismatches
o set object flag when decoding objects that have
a "class" attribute (fixes issues with S3 objects that
were passed from the client to the server).
o set S4 bit for pure S4 objects (S4SEXP). No other S4
objects are supported because there is no way to tell
that an assembled object is really an S4 object
o added string encoding support (where R supports it)
The string encoding can be set in the configuration file
(directive "encoding"), on the command line with --RS-encoding
or within a session by the client command CMD_setEncoding.
This means that strings are converted to the given encoding
before being sent to the client and also all strings from the
client are assumed to come from the given encoding.
(Previously the strings were always passed as-is with no
conversion). The currently supported encodings are "native"
(same as the server session locale), "utf8" and "latin1". The
server default is currently "native" for compatibility with
previous versions (but may change to "utf8" in the future, so
explicit use of encoding in the config file is advised).
If a server is used mainly by Java clients, it is advisable to
set the server encoding to "utf8" since that it the only
encoding supported by Java clients.
For efficieny it is still advisable to run Rserve in the same
locale as the majority of clients to minimize the necessary
conversions. With diverse clients UTF-8 is the most versatile
encoding for the server to run in while it can still serve
latin1 clients as well.
0.5-2 2008-10-17
o fix a bug in CMD_readFile and CMD_setBufferSize that
resulted in invalid buffer sizes (one of the ways to
trigger the bug was to attempt to read a small number of
bytes with readFile). Thanks to H. Rehauer for reporting.
o ignore attributes if they are not in a LISTSXP - there seem
to be other uses of the ATTRIB entry in conjunction with
character hashes in recent R versions. (BR #76)
o adapt C++ client to changes in 0.5 (at least to the point
where the demo1 code works)
o add support for XT_VECTOR_EXP in assignments
o improve protection for vectors
o report "remote" setting in --RS-settings
o updates in the REngine Java client, added documentation
0.5-1 2008-07-22
o fix build issue with R 2.7.x on Windows
o mergefat now works properly and uses cp if there is no lipo
(this fixes multi-arch issues on Mac OS X and makes sure that
Rserve/Rserve.dbg are installed even on non-Mac systems)
0.5-0 2008-07-21
o added CMD_serEval and CMD_serAssign which are highly efficient
when talking to R clients as they don't need any intermediate
buffer. The corresponding R client functions RSeval and
RSassign have been re-written to use this new API.
o deprecate scalar types in the protocol
o add more efficient storage for dotted-pair lists
and symbol names
o add support for complex numbers
o new Java client: REngine
it is more flexible than JRclient and it can be used with
other Java/R engines such as JRI. Also it has a much more
clean API and better exeption handling.
- allow NaNs to be passed in raw form to R, i.e. double
NAs can be created using
Double.longBitsToDouble(0x7ff00000000007a2L)
(nice methods for this should follow)
o C++ client was moved to src/client/cxx
JRclient:
o change the representation of lists to generic
named vectors (class RList)
o change the ways attributes are accessed
0.4-7 2007-01-14
o relax DLL versions checking on Windows
o added more sophisticated implementation of RSassign
in R client to support larger data. Nevertheless, due to
limitations in R, objects must be serializable to
less than 8MB to be assignable via RSassign.
o added more robust error handling in the R client
o fixed compilation on systems with custom include dir
(such as Debian)
o JRclient is now part of the Rserve package.
See clients.txt for details.
It is not compiled by default (but installed when
--with-client is specified), because we cannot assume the
existence of a Java compiler.
0.4-6 2006-11-30
o fixed bug in RSeval when handling large objects
o minor fix in RSassign
o add an endianness hack for Windows in case config.h is not
included properly
0.4-5 2006-11-29
o added --with-server option (by default enabled). When disabled,
the server itself is not built. When enabled, R must provide R
shared library, i.e. it must have been compiled with
--enable-R-shlib.
o added --with-client option (by default disabled). When
enabled, the C/C++ client is built and installed in the
package. It will be copied in the "client" directory of the
package and contains all files necessary for building a
client application.
This option has no effect on the R client which is always
built and installed.
o Windows version of Rserve now builds and installs both debug
(Rserve_d.exe) and regular (Rserve.exe) version of Rserve. In
addition, the Rserve function can now be used to launch Rserve
even on Windows.
o endianness detection now prefers information from the compiler
macros thus allowing cross-compilation. Use -D_BIG_ENDIAN_ or
-D_LITTLE_ENDIAN_ to override it if necessary.
o allows universal build on Mac OS X
o adapt to R_ParseVector interface change in R-devel
0.4-4 2006-11-15
o first release on CRAN
o added support for RAW type (both in and out)
o added rudimentary client support (thanks to David Reiss for
his contributions) and documentation
Previous major releases:
0.4 2005-08-31
* added support for sessions
0.3 2003-10-07
* new format for boolean arrays
last version: 0.3-18 (2005-08-28)
0.2 2003-08-21
* support for large objects
0.1 2002-07-06
* first release