|
|
|||||||||
WANdisco Subversion Replicator makes it possible to have multiple active replicas of a Subversion repository that stay in sync. Each repository acts as if it is the local Subversion server for the user base. This is achieved via the Distributed Agreement Engine technology underlying WANdisco Subversion Replicator.
Subversion is an open source code versioning system. Subversion, like most source code versioning systems, is designed to run as a central server to which multiple Subversion clients connect using SVNor HTTP(s) protocol over TCP. WANdisco Subversion Replicator allows you to break the central paradigm and have multiple replicas of the Subversion repository.
In this administration guide, you will learn how to easily setup WANdisco Subversion Replicator and its Distributed Agreement Engine.
This guide is intended for a Subversion administrator or a user who is reasonably comfortable with:
inetd/xinetd service on Unix/Cygwin or Windows service
If you don't meet the pre-requisites you may want to contact your Subversion administrator or request WANdisco to do a professional install for you.
Before running the WANdisco Subversion Replicator, please ensure:
C:/Program Files/java/jdk1.5.0_03. Choose an alternate installation
directory, for example C:/java.
license.key file into svn-replicator/config.
Depending upon your license,
certain features may be disabled. This document will highlight any features that
require an Enterprise
Edition license. In particular, security (access control) features require an
Enterprise
Edition license.
JAVA_HOME environment
variable is defined.
JDK is installed, not just the JRE. This can
be confirmed by running java -server -version. If it generates a not found
error, uninstall the JRE Java package and reinstall the JDK Java
package.
perl executable is on
the system PATH.
logtimefix script requires Perl package Date::Manip.
- To install a Perl package on UNIX/Cygwin, using CPAN, for example:
$ perl -MCPAN -eshell cpan> install Term::ReadKey
perl executable is on
the system PATH. Please download the MSI installer for ActivePerl from
here: [http://activestate.com/Products/Download/Download.plex?id=ActivePerl]
logtimefix script requires Perl package Date::Manip.
- To install a Perl package on Windows, using ActivePerl package manager,
for example:
c:\>ppm ... ppm>install DateManip
importauditdb script requires Perl package Perl::DBI.
XML::Parser package is required. It should be bundled as a standard module on a perl 5.8 installation.
svnserve deployment, ensure that
the WANdisco Subversion Replicator can communicate with Subversion using the SVN RA protocol.
Although the WANdisco Subversion Replicator uses the SVN RA protocol to communicate with the Subversion
server, the Subversion client can communicate with the WANdisco Subversion Replicator using either svn:// or svn+ssh://
protocol.
httpd deployment, ensure that
the WANdisco Subversion Replicator can communicate with the Apache server using the HTTP protocol.
Untar or unzip (using WinZip for example on Windows) the WANdisco Subversion Replicator package
(tar.gz) into the intended subdirectory.
You should see the following directory structure:
$ cd svn-replicator $ ls config lib logs bin docs systemdb
svnreplicator , shutdown
config/svn
subdirectory contains sample configuration file for svnserve.
jar files and DLLs that are required to run the product.
SVNProxyServer-prefs.log.0.
Most projects already have an existing Subversion repository that needs to be replicated. If you are creating a brand new Subversion repository, please follow the Subversion documentation at [http://svnbook.red-bean.com].
In order to make Subversion repository function in a distributed environment, the WANdisco Subversion Replicator maintains its own replicated Authentication database. This allows an Subversion user to connect to any replica and authenticate with the exact same credentails. In addition a Subversion user password can be changed at all replicas by simply modifying it from any one site. The Subversion administrator doesn't need to worry about keeping passwords in sync across the replication group. Please create new users in the WANdisco Subversion Replicator's authentication database via the web console before using the Subversion client to connect with the WANdisco Subversion Replicator.
Please modify the Subversion configuration under the <svnroot>/conf directory as
following, in order to enable WANdisco Subversion Replicator's replicated Authentication database:
svnserve.conf to disable anonymous access
$ vi conf/svnserve.conf ... [general] ### These options control access to the repository for unauthenticated ### and authenticated users. Valid values are "write", "read", ### and "none". The sample settings below are the defaults. anon-access = none ... password-db = passwd
passwd file (see express setup section below). If auto-update is
disabled, manually modify the passwd file such that password for all users
are the same and match the
password defined inside the XML element SVNProxy.svnPassword in the svn-replicator/config/prefs.xml
file.
If prefs.xml does not specify a svnPassword element, wandisco is used as
the default password.
All the authentication is done by the WANdisco Subversion Replicator itself, so the backend Subversion's
security is not compromised in anyway.
$ cat conf/passwd ... [users] user0 = wandisco ... userN = wandiscoFor your convenience, the WANdisco Subversion Replicator has sample configuration files for the above settings. Please look at the files under
svn-replicator/config/svn:
$ ls svn-replicator/config/svn passwd svnserve.conf
In order to make Subversion repository function in a distributed environment, the WANdisco Subversion Replicator requires exactly the same Apache/SVN setup at all the sites. In addition Apache's configuration needs to be tweaked to work in a peformant manner with Subversion (this is independent of the WANdisco Subversion Replicator) . Please ensure the following:
conf/httpd.conf or included
conf/extra/httpd-defaults.conf. For instance,
$ vi conf/httpd.conf ... # Various default settings Include conf/extra/httpd-default.conf ... $ vi conf/extra/httpd-default.conf ... # # Timeout: The number of seconds before receives and sends time out. # Timeout 300000 # KeepAlive: Whether or not to allow persistent connections (more than # one request per connection). # KeepAlive On # MaxKeepAliveRequests: The maximum number of requests to allow # during a persistent connection. Set to 0 to allow an unlimited amount. # MaxKeepAliveRequests 0 # # KeepAliveTimeout: Number of seconds to wait for the next request from the # same client on the same connection. # KeepAliveTimeout 500000 ...
# Site A
$ cat conf/extra/httpd-svn-dav.conf
...
NameVirtualHost site-a:8181
<VirtualHost site-a:8181>
<Location /dir0>
DAV svn
SVNPath /home/site-a/svnroot
AuthType Basic
AuthName wandisco
AuthUserFile /home/site-a/apache2/dist/conf/htpasswd
Require valid-user
</Location>
</VirtualHost>
...
# Site B
$ cat conf/extra/httpd-svn-dav.conf
...
NameVirtualHost site-b:9191
<VirtualHost site-b:9191>
<Location /dir0>
DAV svn
SVNPath /home/site-b/svnroot
AuthType Basic
AuthName wandisco
AuthUserFile /home/site-b/apache2/dist/conf/htpasswd
Require valid-user
</Location>
</VirtualHost>
...
OPTIONS and PROPFIND. In plain speak, anonymous access to
Apache is not allowed to enforce license requirements unless you have an unlimited or an
evaluation license.
With an unlimited or evaluation license, you are not required to register the user via the
WANdisco Subversion Replicator's Create User web interface. This will typically mean ensuring a Require valid-user
line is specified in the Apache SVN DAV configuration section.
HTTP write methods to be authenticated. If your Apache configuration files make use of Limit or LimitExcept directives to restrict the HTTP methods under access control, please ensure the following HTTP methods always require a valid user: MKACTIVTY, UNLOCK, LOCK, PROPPATCH.
Most Subversion clients support http and https (http over SSL). In order to allow Subversion
clients to connect via HTTPS/SSL to the WANdisco Subversion Replicator, you need to setup an Stunnel daemon.
Stunnel is a program that allows you to encrypt arbitrary TCP connections inside SSL
(Secure Sockets Layer). It requires a functioning SSL library such as OpenSSL or SSLeay in
order to compile stunnel.
Once stunnel is installed, edit the stunnel configuration file to accept connection
on an SSL port (443 for example) and forward to the WANdisco Subversion Replicator's listener port. For instance,
if the WANdisco Subversion Replicator is listening for Subversion http traffic on port 80, the stunnel configuration
would look like:
[https] accept = 443 connect = 80 ...
The WANdisco Subversion Replicator supports either an express setup or a custom setup. If this is the first time setting up the WANdisco Subversion Replicator, we recommend starting with the express setup method.
If the installation requirements as specified in the previous section have been met, the express setup should take 20 minutes or less to get a basic WANdisco Subversion Replicator environment configured.
The express setup option can be used to quickly create the prefs xml
configuration file used by WANdisco Subversion Replicator. This is accomplished by running the bundled
program, svn-replicator/bin/setup. The text console based UI will guide you
through basic configuration options.
At the end of the setup program, you would have created a ``prefs-
{hostname}.xml`` file for each replicator in the replication group. Simply copy
the svn-replicator/config/prefs-(hostname).xml to the svn-replicator/config/prefs.xml file on the
corresponding replicator host.
We will now walk you step by step, through the various setup screens.
setup program:
$ svn-replicator/bin/setup
HTTP or SVN) and the number of replicas
you want to configure. Each replica has an
associated WANdisco Subversion Replicator with it's own prefs.xml configuration file.
=====================================================
Welcome to the WANdisco Replicator's Express Setup
You can use it to setup configuration files used by
WANdisco SVN Replicator.
You will just need to enter the minimum information
required to setup each replicator instance. At the end
of the setup run, you will have a "perfs-<hostname>.xml" for
each replicator. Just copy it to the remote replicator's
config/ directory and start the svnreplicator.
=====================================================
With Subversion you must choose if clients will use HTTP or SVN
protocol for remote access. HTTP protocol implies the http:// scheme;
SVN protocol implies the svn:// scheme.
Use SVN protocol ? [Y|N] [Y] : n
conf/password file.
You do not have to use Apache/HTTP to just get the LDAP integration.
For LDAP authentication, it is required that the passwords are saved as {CLEARTEXT} in LDAP database.
With SVN protocol, WANdisco can support authentication against an LDAP server. You do not need to use Apache/HTTP to get LDAP integration. If you do not wish to use LDAP, you can choose to use the default svn repository conf/password file. Use LDAP for authentication? [Y|N] [Y] : y How many replicas do you want? [2] : 2
Setting up replicator instance .... #1
______________________________________
Now you will specify the Ethernet MAC address of the host on which
Replicator would be running. It is required that you specify a unique
MAC address for each host on which Replicator would be running.
The MAC address on UNIX can be obtained via "ifconfig" command
and on Windows via "ipconfig /all" command. The MAC Address looks like
this - 00-02-A5-C1-7A-2F (Windows) or 00:02:A5:C1:7A:2F (UNIX). If you
don't have all the MAC addresses handy, now would be a good time to
get them before proceeding further.
Enter the MAC Address :
Setting up replicator instance .... #1
______________________________________
Now you will specify the host:port used by SVN clients to
connect with the Replicator. Setting the port to 3690,
would be the most transparent option from the SVN client
perspective. Note you can NOT specify 0.0.0.0 or localhost
as the the host on which Replicator would be running. The
hostname needs to be the DNS hostname or the valid IP address to
which remote SVN clients as well as remote SVN Replicators can connect.
For example, let us say on a subnet 192.168.1 in Tokyo, the LAN address of
Replicator machine is 192.168.1.29 and the external WAN address is
203.23.12.129 (DNS hostname is tokyo.svnrus.org). The Replicator address
should be specified as 203.23.12.129 or tokyo.svnrus.org and NOT 192.168.1.29.
Enter the hostname or IP address of the Replicator#1 : tao
You specified tao, is this correct?[Y|N] [Y] : Y
Enter the TCP port for the Replicator#1 [3690] :
You specified 3690, is this correct?[Y|N] [Y] :
Setting up replicator instance .... #1
______________________________________
Now you will specify the DConeNet port used by the Replicator to
communicate with other Replicators. This is not visible to SVN
clients but used for actual data transfer between the Replicators.
Enter the TCP port for DConeNet [6444] :
svnserve host and port used by the WANdisco Subversion Replicator to communicate with
the backend Subversion repository. After entering this information, the setup
program will generate a GUID for the WANdisco Subversion Replicator being configured. Make sure you choose
a listen port for svnserve that is different from WANdisco Subversion Replicator, if they are running
on the same host. If two instances of the WANdisco Subversion Replicator are running on the same machine,
each of them should connect to a unique listen port for svnserve.
Setting up replicator instance .... #1
______________________________________
Now you will specify the svnpserver host:port used by SVN repository.
This is typically specified in the /etc/services or inetd file on UNIX.
It is recommended you change the default svnserve port to something
other than 3690 to avoid operational errors. If the svnserve
is allowed to ONLY run on localhost, do NOT specify a remote hostname
or IP address then just specify localhost.
The default is localhost which means the Replicator and svnserve
are running on the same host. If that is not the case, please change it.
Enter the svnserve hostname [localhost] :
You specified localhost, is this correct?[Y|N] [Y] :
Enter the svnserve port [3691] :
You specified 3691, is this correct?[Y|N] [Y] :
HTTP protocol or authentication against LDAP, this dialog will not be displayed.
This option is only available if Subversion server and the repository is on the same host
and auto-update is enabled as below.
If none of the above is true, then the administrator needs to add the Subversion user
to the appropriate password file manually as described in [section above
#svnrepo].
Setting up replicator instance .... #1
______________________________________
The setup will now configure the auto-update of Subversion conf/password file.
The update of the password file is optional and is only available if the svnreplicator
and svn repository are on the same host. The auto-update option allows the administrator
to change the user password at all sites using a central Web console. The password update
is replicated using the fault-tolerant DCone engine to all sites. Using this feature,
your remote sites do not have to worry about lost network connectivity to a central
authentication database like LDAP/NIS.
Is svnroot and svnreplicator on the same host? [N] : Y
You specified Y, is this correct?[Y|N] [Y] : Y
Should svnreplicator auto-update the password file? [N] : Y
You specified Y, is this correct?[Y|N] [Y] :
Enter the fully qualified path to the svn password file : /export/svnroot/config/passwd
You specified /export/svnroot/config/passwd, is this correct?[Y|N] [Y] :
Generating a GUID for Replicator#1...b1e7b22c-b9ae-11d9-902a-0002a5c17a2f [Done]
Hit Enter to continue ....
HTTP protocol
or authentication against Subversion password file, this dialog will not be displayed.
Setting up replicator instance .... #1
______________________________________
Now the setup will capture the properties that are required to connect
to an LDAP server. Please contact your LDAP administrator for valid
values.
Example:
LDAP Host : zeus
LDAP Port : 389
rootdn : cn=Manager,dc=example,dc=com
root-password : secret
seach-context : dc=engineering,dc=example,dc=com
object-class : person
Enter the hostname or the IP address of the LDAP server : zeus
You specified zeus, is this correct?[Y|N] [Y] : y
Enter the port the LDAP server is listening on [389] :
You specified 389, is this correct?[Y|N] [Y] : y
Enter the 'rootdn' : cn=Manager,dc=example,dc=com
You specified cn=Manager,dc=example,dc=com, is this correct?[Y|N] [Y] : y
Enter 'root-password' : secret
You specified secret, is this correct?[Y|N] [Y] : y
Enter the search-context ? : dc=engineering,dc=example,dc=com
You specified dc=engineering,dc=example,dc=com, is this correct?[Y|N] [Y] :
Enter the 'object-class' : person
You specified person, is this correct?[Y|N] [Y] : y
Quorum Selection
-----------------
Singleton Quorum optimizes performance at the Distinguished node (which
can be rotated using a cron job or Replicator's scheduler).
Majority Quorum optimizes availability by ensuring a majority of sites
vote on a write transaction first.
Unanimous Quorum requires all sites to be up and running to commit a
transaction. See the DCone Administration Guide for further information.
You can select from the quorum list below:
1. Singleton
2. Majority
3. Unanimous
Choose a quorum [1] :
Which of the following Replicators will be the initial Distinguished Node?
1. svnhost-usa.com
2. svnhost-uk.com
Select the distinguished node [1] :
prefs xml files. The setup
program will create the prefs-{hostname}.xml configuration files.
Now you will specify the destination directory where prefs xml file for
each replicator will be generated. Remember to copy the generated
prefs-{replicator name}.xml file to the ``svn-replicator/config`` directory
on each Replicator's host machine. After copying, make sure you
*rename* the file to "prefs.xml". If you forget to rename the
configuration will not take effect!
Enter the directory where prefs xml files will be saved [../config] :
config directory on each host
and rename them to prefs.xml. Now you are ready to run the WANdisco Subversion Replicator.
Express setup tool supports a -silent and -record option to allow
an admin to perform a silent install without being prompted for input
on the console.
An admin could start the setup program in the record mode
and then latter use the recorded answers file to replay the
answers and perform a silent install. The admin could modify
the recorded answers in a text editor and then use -silent
to create new configuration files. For example
$ ./svn-replicator/bin/setup -record my-answers $ vi my-answers $ ./svn-replicator/bin/setup -silent my-answers $ ./svn-replicator/bin/setup -silent old-ans -record new-ans
The answers are recorded continuously, so if you restart setup you can also use the recorded file to pick up from where you left off, without having to re-enter the answers.
For more information look at the usage of the setup command:
$ ./svn-replicator/bin/setup -h
setup [-silent recorded-setup-file] [-record file-to-record-to]
-silent recorded-setup-file :
Silent install will use the supplied "recorded-setup-file" to
automatically answer the setup interview questions. If all the
answers are not supplied, it will prompt on the console.
-record file-to-record-to
Will record all the valid interview answers to the
"file-to-record-to". Can latter be used for silent install.
Both options can also be used at the same time. For
example to continue an install from where you last
left off you could do:
setup -silent prev-silent-file -record new-silent-file
In order to use this feature, you must have recorded the answers when you did your initial setup.
Consider the following scenario: The administrator wants to change a 3 site installation to a 4 site installation. In other words bring one more site online.
The administrator can use the silent install as following:
nreplicas="3"
or changing the number from 3 to 4.
$ setup -record answers3sites ... $ vi ./answers3sites nreplicas="3" <== Change 3 to 4 or comment out the line by putting # at the start mac.1="00:08:5B:19:3E:F5" repHost.1="pao" repPort.1="2401" dconePort.1="6444" ...
$ setup -silent answers3sites
answers3sites file. The administrator does not
have
to re-answer questions that have already been answered.
This section describes how to customize the configuration generated by Express Setup.
The scenario assumes you are setting up a two node configuration as below:
To get this scenario working quickly all you need is :
svnserve is listening on
Open the svn-replicator/config/prefs.xml file in your favorite editor and modify:
Modify the listenerHost:listenerPort in the svn-replicator/config/prefs.xml file to
the host:port on which the WANdisco Subversion Replicator will listen for incoming Subversion
client traffic. If you do not want the Subversion client to modify
it's svn URL, just ensure that the listenerHost:listenerPort
stays the same as the current svnserve settings.
Next, point the svnHost:svnPort to point to the
host:port on which svnserve is going to listen to.
If using xinetd.d for launching svnserve, you may have
to modify a file with a name like /etc/xinetd.d/svnserve to change the
default port (3690) on which svnserve will listen on. If using Apache
server, the default port (80 or 443) may have to be changed. This is required to avoid port conflicts when replicator and svnserve or Apache
are running on the same box.
Here is a sample from svn-replicator/config/prefs.xml that you can use as an example:
<SVNProxy>
<listenerHost>svn-host</listenerHost>
<listenerPort>3690</listenerPort>
<svnHost>svn-server</serverHost>
<serverPort>3690</serverPort>
<useHttp>false</useHttp>
</SVNProxy>
In the above, the same port (3690) was kept for both the
WANdisco Subversion Replicator and Subversion svnserve as they are running on different hosts.
If that wasn't the case you will have to modify the svnserve
port if you want the Subversion clients to retain their svn URL setting.
If changing the xinetd.d file, you would typically change
the port as following:
service svnserve
{
disable = no
flags = REUSE
socket_type = stream
wait = no
user = root
log_type = FILE /var/log/svnserve
protocol = tcp
log_on_failure += USERID
# port = 3690
# change port to avoid conflict with WANdisco Subversion Replicator's listening port : 3690
port = 3691
server = /usr/bin/svnserve
server_args = -i
}
Sometimes, administrators specify the Subversion server port in /etc/services.
If you made any changes to /etc/services or the xinetd/inetd
configuration files like svnserve, be sure to restart inetd/xinetd
itself. Otherwise the changes will not take effect and you will have trouble
connecting with the svnserve.
You will install and setup a WANdisco Subversion Replicator for each replica in your configuration. Each WANdisco Subversion Replicator needs to be configured to connect with other replicators. We are essentially establishing initial //group membership//.
Let us say you need to create two replicas - one in San Francisco and one in Bangalore. You will install WANdisco Subversion Replicator at each location. Then you will point them at each other as followings:
guidgen utility that is bundled with WANdisco Subversion Replicator to
instantiate two GUIDs for the San Francisco and Bangalore data centers.
$ guidgen 2 GUID 0. = c9bcec19-4301-11d9-b12d-080020b0d8b8 GUID 1. = 3bfbf219-2918-11d7-80c5-00065be02953
<ProviderDescriptor>
<DefaultProviderId>3bfbf219-2918-11d7-80c5-00065be02953</DefaultProviderId>
</ProviderDescriptor>
<MemberList>
<Member name="3bfbf219-2918-11d7-80c5-00065be02953">
<Profiles>
<TransportPolicy>AlwaysDConeNet</TransportPolicy>
<Transport>
<DConeNet>
<ListenerIP>sanfranciso-replicator</ListenerIP>
<ListenerPort>6020</ListenerPort>
</DConeNet>
</Transport>
</Profiles>
</Member>
<Member name="c9bcec19-4301-11d9-b12d-080020b0d8b8">
<Profiles>
<TransportPolicy>AlwaysDConeNet</TransportPolicy>
<Transport>
<DConeNet>
<ListenerIP>bangalore-replicator</ListenerIP>
<ListenerPort>6020</ListenerPort>
</DConeNet>
</Transport>
</Profiles>
</Member>
</MemberList>
The first configuration parameter is a DefaultProviderId. This associates a
GUID with the specific data center at which we are setting up the replicator.
In the above svn-replicator/config/prefs.xml file, we are associating the GUID ``3bfbf219-2918-11d7-80c5-
00065be02953`` with the San Francisco data center. For Bangalore data center we
will associate the GUID c9bcec19-4301-11d9-b12d-080020b0d8b8 by
specifying it in the DefaultProviderId setting.
Next, we specify the Distributed Agreement Engine host:port for each data center. You can choose
any static port. The ListenerIP field could specify a host name or
an IP address. Now, when San Francisco data center starts up it
knows how to communicate with the Bangalore data center and vice versa.
That's it, now you are ready to run the WANdisco Subversion Replicator, using the startup script provided. To run it from the command line:
$ svn-replicator/bin/svnreplicator & $ tail -f svn-replicator/logs/SVNProxyServer-prefs.log.0 ....
If you selected the SVN protocol, you will see:
INFO: [main] SVN Proxy listener is now turned ON at port :3690
If you selected the HTTP protocol, you will see:
INFO: [main] WebDAV Proxy listener is now turned ON at port :80
When you see the last line, you know WANdisco Subversion Replicator has started successfully. Alternatively, you can go to the web console and check the dashboard.
To shutdown WANdisco Subversion Replicator, just run
$ svn-replicator/bin/shutdown
Caution: To ensure that the Subversion repositories do not go out of sync, we recommend taking all possible precautions to avoid direct access to the Subversion server, bypassing the WANdisco Subversion Replicator. For example, you could setup Subversion server to only allow connection from the IP address of the host on which Replicator is running, you could limit shell access to the replicator and Subversion machine.
Please ensure your license.key file specifies valid number of users, sites
and allowed IP addresses on which the WANdisco Subversion Replicator is allowed to run. If you have an
unlimited license, you do not have any restrictions on number of users,
sites or IP addresses.
In order for Subversion users to access repoistory via WANdisco Subversion Replicator, the administrator must have
registered valid users using either the Web console or the import facility (see
User Management section under Access Control chapter). When using the HTTP
protocol, this requirement is waived with the purchase of an unlimited license.
The WANdisco Subversion Replicator evalutaion license is an unlimited standard Subversion Edition license with a limited expiration time.
Please ensure the following prior to running the svnreplicator:
svnroot directories on all the replicas match. Make sure
the initial contents are exactly the same including the repository UUID. Easiest
way would be to create a tar or a zip archive of the current repository and use
that to create all the other repositories.
HTTP protocol, Apache major/minor version, mod_dav and mod_svn_dav versions should also match.
Here are common gotchas and their solutions:
logs/SVNProxyServer*.log.* to know if
the replicator even got the write or commit changes:
... INFO: [reader-2] Submitted after verification: svn-proposal-2f195a4b-dea9-11d9- b140-00065b193ef5_4 1118955513564 org.nirala.communication.transport.svnproxy.SVNProposal output INFO: [gsn-1] Executing svn-proposal-2f195a4b-dea9-11d9-b140-00065b193ef5_4 1118955513575 org.nirala.communication.transport.svnproxy.SVNProposal onCommit INFO: [reader-1] Committed SVN transaction svn-proposal-2f195a4b-dea9-11d9- b140-00065b193ef5_4 ...If you don't see a
Submitted line in the log file, the Subversion write operation
never made it the the WANdisco Subversion Replicator and hence can't be replicated. On completing a write
or a commit operation, the WANdisco Subversion Replicator will print a Committed line in the log file.
Submitted lines in the log file after doing commits
Most common reason for this is incorrectly setup Subversion URL. For example if
the Subversion URL is pointing to the Subversion server directly such that the WANdisco Subversion Replicator is
bypassed, then no changesets will be replicated. Best practise is to ensure the
Subversion svnserve is only connected to via the WANdisco Subversion Replicator.
The Subversion URL needs to be setup as:
svn://<replicator-host>:<replicator-port>/<svnpath..> OR http://<replicator-host>:<replicator-port>/<svnpath..>If the replicator is running on Subversion port
3690 or 80 then replicator-port setting
can be skipped on non-localhost machines. If the replicator and Subversion
server are co-located, you need to specify the port when running on the Subversion
server host. This is to ensure the port specification for svnserve or httpd in
/etc/services doesn't cause the Subversion client to bypass the replicator.
$ svn-replicator/bin/reset
<SVNProxy>
<AutoBypassGlobalFailures>true</AutoBypassGlobalFailures>
</SVNProxy>
If you choose to do this, note the above caveat regarding the consequences of
Subversion bugs.
However, if the Subversion transaction failed at some sites but not at others, the WANdisco Subversion Replicator can not bypass the transaction, since that would cause the repositories to be out of sync. These transactions are listed on the dashboard as problem transactions.
The cause of the fatal error can be determined from the log files. After addressing the cause, (for example, if a disk-full error was encountered, add more disk) restart the WANdisco Subversion Replicator. Finally, verify from the dashboard that there are no outstanding problem transactions.
svn-replicator/bin/talkback at each WANdisco Subversion Replicator site and upload the generated tar ball.
If you can not run the talkback tool, send us the current logs files in
logs/SVNProxy*.log at each site and the configuration files (prefs.xml
and /etc/prefs.conf). We may request you to gather more information using
the WANdisco Subversion Replicator web administration interface if needed.
The WANdisco for Subversion Enterprise Edition enables an administrator to implement a comprehensive security model that provides features beyond those of the SVN server.
Note: The WANdisco for Subversion Enterprise Edition currently requires the use of Subversion svn remote access protocol. This release does not support WANdisco for Subversion Enterprise Edition with http access protocol. If you are using http, please use Apache's access control mechansim for authorization.
The Group Management functions allow an administrator to:
The WANdisco Groups are hierarchical with parent-children associations between sub-groups. Groups provide a convenient way of organizing multiple users into a related category for controlling access as well as searching for users. It is recommended that you setup ACLs on a group basis rather than users (both are allowed) for easier management of security policies.
The first step towards setting up security is to map organization or project structure to WANdisco Groups (also referred to as Roles in security literature). The groups are hierarchical. If a user belongs to a parent group, they children groups automatically inherit users in the top-level parent groups. In the same way, Access Rules or ACLs attached to children group are automatically applied to the parent groups. Before creating new groups it is important to work out the parent-child hierarchy. For instance an engineering team may be split up along geographical sites: sanramon-engr, tokyo-engr. These two groups may belong to a parent group: engr. Within sanramon-engr group there may be role based hierarchy: sanramon-project-projectlead, sanramon-qa.
To add a new group, click on the Create Group option in the security menu and specify its name. The name can contain any character including white-space, there are no restrictions. The group name is the primary key into the group database, it can not be changed. If you need to change it just delete the group and add a new one with a different name. Description field can contain any relevant text describing the group. The WANdisco for Subversion Enterprise Edition tracks the creation and modification time-stamp on the groups automatically.
Once a group is created, you can start assigning users to it.
Please note that a system group Admin always exist in the WANdisco for Subversion Enterprise Edition. As mentioned in the ACL section below, a user belonging to the Admin group gets Admin privileges which include List,Read,Write, Copy, Delete privileges on all resources.
The group is assigned under the checked group as a sub-group. To de-assign, uncheck the checkbox and click 'Save Details'. Note by selecting a group, this group is automatically assigned as its sub-group. A sub-group can have exactly one parent. Clicking on the icon e will allow you to edit the specific group.
From the security web console, click on an edit Group icon to go to the Edit Group page. The edit Group icons are visible from any Group management page. Click on the delete button to remove a group.
When a group is deleted it is also removed from the all users who previously belonged to that group. The ACLs associated with the groups are not deleted, they no longer apply though. You can edit them and assign them to a different group.
On the Web console, the Add or remove function under Group management is context-aware. When you select a group by clicking on its checkbox, only users that can be added or removed are shown. This ensures you don't have to worry about duplicate users when adding them to a group. You can use it to quickly add/remove multiple users to/from a given group or sub-group.
If a user has been added to a parent group, they automatically belong to any sub-groups underneath it. So there is no need to add them to the sub-groups. The context-aware web UI will show users who are not in parent groups for example when adding them to a sub-group.
To quickly view a list of users who belong directly to a specific group, just click on the "List users only in this group" in the Edit Group page. To view the list of all users including the ones inherited from ancestor groups, click on the "List all users including inherited from parent groups" hyperlink.
Go to the Edit Group page for the group. Scroll down to the "Resource Details" panel. The resource table contains the listing of all the files and directory patterns (regular expressions) and corresponding ACLs applicable to the specified group. You can click on the ACL hyperlink to edit the associated rule.
The User Management functions allow an administrator to:
The users must exist in the authentication database (svnroot/config/passwd ) used by the SVN server. The User management function allows administrator to specify licensed users who can access the SVN repository via the WANdisco for Subversion Enterprise Edition. Even if you are not using security features like ACL, you must register the valid users with the WANdisco for Subversion Enterprise Edition.
A user who has not been registered will experience an SVN authentication failure.
To add a new user, click on the Create User option in the security menu and specify a login id and password. The last and first name can contain any character including white-space, there are no restrictions. You can specify an optional email address for the user. The user id is the primary key into the user database, it can not be changed. If you need to change it just delete the user and add a new one with a different user id. The WANdisco for Subversion Enterprise Edition tracks the creation and modification time-stamp on the users automatically.
To remove a user, just click on "Delete" button in the Edit User page. You can also delete multiple users from the User List page.
The user can be assigned to any number of checked groups. To de-assign, uncheck the checkbox and click 'Save Details'. Note by selecting a group, the user is automatically assigned to the group and all its sub-groups. Clicking on the icon e will allow you to edit the specific group.
To get a list of all the registered users, just click on the User List option. The user list page shows all the users by default. The page size by default is set to show 25 users per page. You can change that by selecting the page size on the upper right corner. The page control on the left corner allow you to go to next or previous page.
If you want to locate a user by first or last name quickly, just start typing the name in the User name like textbox and an incremental search starts automatically. To get the full list back just hit the backspace button on the keyboard till you have deleted all the characters you typed in the textbox.
All the columns in the user list are enabled for sorting. By clicking on the column header you can sort the list in ascending or descending order. The sortable columns include: CVS user id, last name, first name, email, last modified date.
To restrict the list by a group or role, just select a group name from the drop down menu. To see users in the selected group as well as all the ancestor groups click the Inherited Users checkbox.
You can click on the user id hyperlink to edit the user. Mass delete action is supported via selecting the checkbox in the table header (next to User Login id column) and then clicking the delete button.
The Access Control or authorization mechanism in WANdisco for Subversion Enterprise Edition is based on the concept of a set of valid principals with adequate privileges to access a secured resource. The ACL management features allow an administrator to:
Here are the definitions of the commonly used terms when describing the WANdisco for Subversion Enterprise Edition's ACL mechanism:
a/b/c/, not a/b/c.
The privileges are ordered. In other words if a principal has Admin privilege, they also have the other three: List, Read, Write, Copy, Delete. If a principal has Write or Copy or Delete privilege, they do not get Admin privilege but have the other two: List, Read. If a principal has Read privilege, they do not get Admin and Write privileges but have the List privilege. If a principal has only List privilege, they do not get Admin, Write, Read privileges.
The following is the mapping of actual SVN commands to minimum privilege needed to execute them:
| SVN Command | Privilege Needed |
|---|---|
| info | List |
| log | List |
| ls | List |
| status | Read |
| cat | Read |
| diff | Read |
| checkout | Read |
| cleanup | Read |
| export | Read |
| update | Read |
| revert | Read |
| annotate | Read |
| propget | Read |
| proplist | Read |
| commit | Write |
| import | Write |
| add | Write |
| lock | Write |
| unlock | Write |
| delete | Delete |
| copy | Copy |
The WANdisco for Subversion Enterprise Edition ships with no rules in the rules database. That implies by default everyone is denied. This is essential for security - it closes the window of vulnerability that would have allowed everyone full access between the time the product is first installed and the admin creates access rules. In order to grant access, the administrator has to explicitly create allow rules.
Using the ACL editor from the web console, the admin can create allow or deny rules.
Powerful Perl style regular expressions can be used wherever patterns are allowed. Principal (user/group) or IP patterns for instance - engineering.* (note the dot) or 217.[0-9]+ are all valid patterns. By default the HEAD branch is specified but you can enter a regular expression just as well - release9.0_.* for instance. Note: With the Perl regular expression syntax, if you need to use the '.' (dot) character literally, you need to escape it with a backslash, otherwise '.' (dot) will match any character. To learn more about regular expressions look at a tutorial here.
Multiple rules can be edited atomically using the WANdisco for Subversion Enterprise Edition. When you submit changes to ACLs, the WANdisco for Subversion Enterprise Edition guarantees either all the rules are updated or none at all. This ensures consistent rules database across multiple sites, with any pattern of failures.
When setting up a rule on a specific directory, please note that the directory name is treated as a regular expression pattern. For example if you want to allow write access to all the files under a directory /svnroot/trunk/docs you will need to specify one of the following patterns:
/svnroot/trunk/docs|/svnroot/trunk/docs/.* or
/svnroot/trunk/docs.*
The second pattern would allow access to all files including, /svnroot/trunk/docs, /svnroot/trunk/{docs, docs-maker, docs2} etc. The first pattern would allow write into the directory (to create new files or directories) as well as all files under the .../docs/ subdirectory.
Special considerations apply for list and read access rules. Unlike write operations, the read and list operations can traverse directory hierarchy. Therefore it makes sense to always allow/deny read and list privileges on all files under a directory. This can be done by specifying a wild-card pattern, for example: allow read from /svnroot/trunk/module1|/svnroot/trunk/module1/.*.
The copy privilege is specified on the source directory. It specifies if a user is allowed to copy from a given directory. The parent directory of the destination of a copy command should have write privilege enabled. Granting write privilege does not imply the user has delete or copy privilege. This allows the administrator to control who can create tags or branches and who can delete version controlled files. For example to allow copy from /trunk to /tags/rel1, you can create two access rules:
/trunk.*
/tags/rel1
The following steps are followed by the WANdisco for Subversion Enterprise Edition's ACL engine when making an allow or deny decision on a SVN command the user is trying to execute:
In summary, the rules applicable to a specific user override the rules applicable to a group.
The following properties in the prefs.xml file can be used to control the ACL engine:
<Security>
<AccessControl>
<Enable>true</Enable>
<Replicate>true</Replicate>
<ClientTimeout>15s</ClientTimeout>
</AccessControl>
</Security>
By default, the WANdisco for Subversion Enterprise Edition has access control enabled. To turn it off set /Security/AccessControl/Enable to false.
The principal (user/group) and rules database are replicated across all sites by default, to not replicate and instead use a site-specific local database, just set /Security/AccessControl/Replicate to false.
If the principal/rules databases are replicated and a change is submitted via a web client, the /Security/AccessControl/ClientTimeout specifies the time after which the web client will timeout while waiting for a confirmation from the WANdisco for Subversion Enterprise Edition. Default timeout is 15 seconds. This is only an issue if the quorum of sites are not available (network down) when committing the change. The changes are kept in a pending transaction journal and applied later when the quorum is restored.
In order to setup security policies the administrator needs to setup the users in the WANdisco Subversion Replicator's internal database. For large organization this can be cumbersome. Often, the user information is already maintained in an external database like LDAP or NIS. For such a deployment, the WANdisco Subversion Replicator for Subversion provides integration with an LDAP/NIS based authentication databases.
The integration allows the administrator to automatically synchronize the user properties (user id, password) from an LDAP/NIS database. The WANdisco Subversion Replicator for Subversion will periodically (default is every 5 minutes) connect with the LDAP/NIS database server and pull the user information. Using the WANdisco web console, the administrator can go and select the new users that were discovered during the synchronization with LDAP/NIS. The selected users are then added to the WANdisco Subversion Replicator's user database. If 'Update Passwords' flag is set and auto-update of $SVNROOT/conf/passwd is turned on.
The following properties can be configured from the WANdisco Subversion Replicator's web console to setup LDAP/NIS integration:
The following properties need to be defined (* indicates a mandatory property) to setup integration with LDAP:
Note: The password should be stored in clear-text on LDAP server as required by Subversion only if 'Update Passwords' flag is set and auto-update of $SVNROOT/conf/passwd is turned on.
If setting up integration with NIS, the following properties need to be setup (* indicates mandatory):
# Column syntax - # 0 seq | 1 time | 2 txid | 3 cmd | 4 user | 5 ipaddress | 6 access | # 7 dir | 8 file | 9 revThe column description is as follows:
| Column Number | Description |
|---|---|
| 0 | Record Sequence Number |
| 1 | UNIX Timestamp |
| 2 | Transaction Id |
| 3 | SVN Command Name |
| 4 | SVN User id |
| 5 | IP Address of User |
| 6 | Access Decision (Allow or Deny) |
| 7 | SVN Directory being accessed |
| 8 | SVN File being accessed |
| 9 | User's File Revision |
The audit trail files are created under cvs-replicator/audit directory. By default they are automatically rotated upto 10 times when they get to 10MBytes. These defaults can be changed via configuration in config/prefs.xml file:
<Audit>
<MaxFileSize>10485760</MaxFileSize>
<MaxFileCount>10</MaxFileCount>
<Disable>false</Disable>
</Audit>
The MaxFileSize specifies a size in bytes, MaxFileCount specifies how many files to rotate before recycling the files. By default auditing is enabled in WANdisco for Subversion Enterprise Edition, it can be turned off by setting Disable to true. To ensure no audit records are lost, please schedule a job (using cron for example) to import the audit records into a SQL database periodically. Inserting in a SQL database also enables complex SQL queries to be made against the audit database.
To create audit files in a different directory just create a symbolic link (cvs-replicator/audit) to another directory.
The WANdisco for Subversion Enterprise Edition bundles a tool importauditdb that allows audit records in audit-trail files to be imported to a SQL database. The usage of the import command is as below:
[admin@smp1 ~/svn-replicator]$ bin/importauditdb -h
Usage:
importauditdb [-host <db-host>] [-port <db-port>] [-user <db user>]
[-pass <db user password>] [-db <database to use>]
-f file-pattern1 file-pattern2 .. file-pattern-N
Defaults:
host : localhost
port : Default DB Port
user : root
password : empty
Database : wd_audit_db
The import tool requires Perl::DBI module to be installed. Please run svn-replicator/bin/checkdbi
to check if the module is installed and correct database driver is available on your system.
Note: Before using import you must create a database (default name is wd_audit_db) on the database server. The import tool will automatically create the table schema in that database, the first time it runs. The import tool uses standard SQL syntax, it makes use of a system function FROM_UNIXTIME, please ensure your database version supports it. MySQL, Microsoft SQLServer support this function.
The replicator by default verifies if a network quorum is reachable when a write command is submitted. If the quorum is un-reachable, by default the write command is aborted and the following message appears on the Subversion client console:
Check the Network connectivity, failed to reach a minimum quorum of nodes. Aborting the svn write operation.
To turn off the quorum check, just set the parameter, AlwaysVerifyQuorum to false in the svn-replicator/config/prefs.xml file. For instance,
<SVNProxy>
<AlwaysVerifyQuorum>false</AlwaysVerifyQuorum>
....
</SVNProxy>
If the check is turned off and quorum is un-reachable, the write transaction will be applied to the WANdisco Subversion Replicator's transaction journal and stay in a pending state till network connectivity and quorum is restored. Note: With singleton quorum, if the current site is also the distinguished node, the quorum check will always succeed irrespective of network connectivity to other sites.
The replicator by default waits for 60 seconds when a write transaction is initiated by the Subversion client and then issues a message to the Subversion client console if the transaction has not completed in 60 seconds. The transaction time may exceed 60 seconds due to couple of reasons - slow network, large checkins, local replicator that the client is connecting to may be back logged. In other words the client after getting the deferred write message is disconnected and the replicator will attempt to apply the changes in the background. This doesn't guarantee that the write will not be aborted by the Subversion server due to "upto-date check failed" condition. All it means is the user Subversion sandbox doesn't need to block. This default behavior can be optionally turned off.
The message that the replicator puts out when it detects a slow write operation (exceeds the default 60 seconds timeout) :
WANdisco Subversion Replicator has accepted the change-set. Your command will be applied to the Subversion repository in the background. After your command has been applied, do a "svn update" to make your sandbox consistent with the Subversion repository. Your transaction ID is NNNN
This behavior can be controlled via the following parameters in the svn-replicator/config/prefs.xml file.
<SVNProxy>
..........
<!-- Defaults to true -->
<UseDeferredWrites>true</UseDeferredWrites>
<!-- Defaults to 60s or 60000 milliseconds -->
<ClientWaitTimeout>60000</ClientWaitTimeout>
</SVNProxy>
The Subversion user can check the completion status of their transaction by going to the
WANdisco Web Console, entering the transaction ID, the site at which the transaction
was initiated and clicking the Track button. The 0.0.0.0 IP address indicates the local site.
If the WANdisco Subversion Replicator has been restarted since the transaction was submitted or command replication has not commenced, the detailed status for the transaction will not be available. The Replication Details specify the completion status of the Subversion command at each site. If the command's completion status is Executed, that implies the command finished with the actual Subversion status as either Successful (committed for a svn commit command) or an error from Subversion (for example, up-to date check failed). If the command has not yet executed, the completion status could be In-Progress or Scheduled. The status In-Progress indicates the command is replicating. The status Scheduled indicates replication is finished and the command is waiting for locks to be obtained in order to execute.
At this time, the WANdisco Subversion Replicator does not report the completion time for the remote sites. The completion time field is only valid for the local site at which the transaction was submitted. The completion status is reported for each site. The Subversion user can use that to tell if the command has completed at remote sites.
When running the WANdisco Subversion Replicator in a production environment it is often necessary to have the replicator process restart on termination or have the WANdisco Subversion Replicator automatically start on machine reboot.
The WANdisco Subversion Replicator has a built-in watchdog which can monitor the replicator process and restart it on termination. By default the watchdog mode is turned on. In the watchdog mode, the svnreplicator process will automatically disassociate from the terminal and become a daemon process so don't background it using &.
To turn off the watchdog mode specify the -nowdog option.
If the WANdisco Subversion Replicator is unable to start up; i.e., if it terminates several times in quick succession, the watchdog will start the WANdisco Subversion Replicator in read-only mode.
When started in this mode,
$ ./bin/svnreplicator -h
Usage: svnreplicator [-v] [-verbose] [-nowdog] [-pause time]
[-email email-address]
-v Print the svnreplicator version
-verbose
Verbose, console messages go to STDOUT/STDERR
instead of logs/console.txt
-nowdog
Turn off watchdog mode, svnreplicator will not start a
watchdog which can automatically restart svnreplicator
process if it terminates. Use this option for testing.
-pause
Time in seconds that the watchdog will pause for, before
restarting service. Defaults to 0 seconds.
-email
Specify an email address to send an alert to, whenever the
Watchdog restarts/shuts down the svnreplicator service.
For email alert to work, sendmail or sendmail wrapper must be
installed in /usr/sbin/sendmail. You may want to test it by
running "sendmail -t" first.
As noted above the admin can specify an -email option to generate email alerts on WANdisco Subversion Replicator restarts.
For instance:
$ svn-replicator/bin/svnreplicator -pause 5 -email admin@svnjungle.com
In order to have the WANdisco Subversion Replicator automatically started on system reboots, the admin can setup a init.d script at an appropriate run-level on UNIX. Here is a sample script on RedHat or Fedora Linux:
#!/bin/bash
#
# chkconfig: - 91 35
# description: Red Hat Linux dist compatible rc script for \
# starting/stoppping a WANdisco agent like cvs|svn replicator etc
#
# Copyright (c) 2006-2016 WANdisco,Inc. Pleasanton,CA,USA
# All rights reserved.
#
# Author: WANdisco Support Staff <support@wandisco.com>
#
# To setup boot time start : chkconfig --level 345 rh-repd on|off
# chkconfig --list rh-repd
# Source function library.
. /etc/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ ${NETWORKING} = "no" ] && exit 0
# Start: Please change to your env
USER="wandisco"
WD_AGENT_TYPE="svn"
WD_AGENT_NAME="${WD_AGENT_TYPE}replicator" # or cvssecurityagent or failoveragent
WD_DISPLAY_NAME="WANdisco Replicator daemon"
WD_AGENT_HOME="/home/${USER}/${WD_AGENT_TYPE}-replicator" # or cvs-security or svn-security
WD_AGENT_OPTS="-wdog -email svnadmins@mycompany.org"
export JAVA_HOME="/export/share/apps/jdks/1.5.0"
# Stop: Please change to your env
WD_AGENT_BIN="${WD_AGENT_HOME}/bin/${WD_AGENT_NAME}"
WD_AGENT_CONF="${WD_AGENT_HOME}/config/prefs.xml"
PID_FILE="${WD_AGENT_HOME}/logs/my.pid"
JAVA_PID="${WD_AGENT_HOME}/logs/java.pid"
NOFILES="65000"
pidfile="my.pid"
checkconfig() {
if [ ! -x ${WD_AGENT_BIN} ]; then
failure "No ${WD_AGENT_BIN} present"
fi
prog=${WD_DISPLAY_NAME}
}
start() {
checkconfig
echo -n "Starting $prog:"
ulimit -S -c 0 >/dev/null 2>&1
ulimit -n ${NOFILES} >/dev/null 2>&1
RETVAL=0
runuser ${USER} -c "${WD_AGENT_BIN} ${WD_AGENT_OPTS}" >/dev/null 2>&1
RETVAL=$?
[ "$RETVAL" -eq 0 ] && success "$WD_AGENT_NAME startup" || \
failure "$WD_AGENT_NAME start"
echo
[ "$RETVAL" -eq 0 ] && touch /var/lock/subsys/$WD_AGENT_NAME
}
stop() {
checkconfig || return 1
echo -n "Shutting down $prog:"
runuser ${USER} -c "${WD_AGENT_HOME}/bin/shutdown" >/dev/null 2>&1
RETVAL=$?
[ "$RETVAL" -eq 0 ] && success "$WD_AGENT_NAME shutdown" || \
failure "$WD_AGENT_NAME shutdown"
echo
[ "$RETVAL" -eq 0 ] && rm -f /var/lock/subsys/$WD_AGENT_NAME
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
stop
sleep 3
start
;;
condrestart)
if [ -f /var/lock/subsys/$WD_AGENT_NAME ]; then
stop
sleep 3
start
fi
;;
*)
echo $"Usage: $0 {start|stop|restart|condrestart}"
exit 1
esac
Please check the download area on our support site (http://support.wandisco.com) for scripts like the above for other platforms (Red Hat Linux etc).
During the normal course of running WANdisco Subversion Replicator, temporary files are generated.
These have the prefix svn-proposal-{GUID}_{seqnum}. By default, they are
written to svn-replicator/systemdb/ directory. This can be over-ridden
using the svn-replicator/config/prefs.xml file as following:
<DirPrefixMap>
<fp->/home/svn/replicator/tmp/dir</fp->
</DirPrefixMap>
These files are periodically garbage-collected by Distributed Agreement Engine at a configurable interval. For more details see the Distributed Agreement Engine Administration Guide
An extremely light-weight HTTP/1.1 web server is built into the WANdisco Subversion Replicator
infrastructure. There is no need to install or manage a separate web
server to acts as a HTTP front-end to the WANdisco Subversion Replicator.
To connect to the HTTP server, simply point the browser to the Distributed Agreement Engine port.
For example, if the Distributed Agreement Engine settings are as follows:
....
<DConeNet>
<ListenerIP>sfo-svn</ListenerIP>
<ListenerPort>6020</ListenerPort>
</DConeNet>
...
The web server can be accessed via a URL similar to [http://sfo-svn:6020]. The Distributed Agreement Engine port is multi-protocol enabled. A single port supports a variety of connections - http, DFTP, DConeNet.
The Web interface can be used to run administrative commands on the local replicator, as well as view status of the WANdisco Subversion Replicator.
The WANdisco web console requires a super user login with a password to do sensitive operations like shutdown. When you first install the WANdisco Subversion Replicator, default super user name is set to root and password is set to wandisco. You should immediately change the default settings. The default super user name can be changed in the svn-replicator/config/prefs.xml file:
<Security>
<Admin>
<user>super</user>
</Admin>
</Security>
The default password can be changed using the web console password form directly. The WANdisco Subversion Replicator never stores the actual password but a one-way hash of the password string in it's internal password database.
You can also control which web console URLs require authentication. The Includes and Excludes elements in the svn-replicator/config/prefs.xml file can be used to specify white-space separated Perl regular expression patterns to mark which URLs require security and which can be accessed without authentication. For example:
<Security>
<Http>
<SecureURL>
<Includes> ^/$ ^/write/.* </Includes>
<Excludes> ^/cvs/dashboard ^/read/.* </Excludes>
</SecureURL>
</Http>
</Security>
To secure the web traffic from the WANdisco web console to the WANdisco Subversion Replicator, you can setup an SSH or SSL tunnel from the web client machine to the WANdisco Subversion Replicator machine. For example:
$ ssh -p 222 -L80:localhost:6444 svn-replica-machine
The above SSH tunnel will ensure passwords and sensitive configuration data never travels in clear-text.
From the web interface, the WANdisco Subversion Replicator dashboard can be accessed. It can be used to view all the vital stats of each replicator site from a single console. The dashboard also shows the last few Subversion transactions that have been committed at each site. The default is 5 transactions, it can be changed using the MaxTxnTrackInDashBoard setting in the svn-replicator/config/prefs.xml file. For example:
<SVNProxy> ... <MaxTxnTrackInDashBoard>20</MaxTxnTrackInDashBoard> ... <.SVNProxy>
The dashboard indicates the status of the rotating quorum. The status is indicated by the color of the word Site in the top left
corner of the frame. The color is green for the distinguished node, and black for the non-distinguished node. When the distinguished node is in transit, the
old distinguished node is indicated in orange, and the new one in red.
WANdisco Subversion Replicator comes bundled with several utilities for monitoring and status checking. These include:
logtimefix on the SVNProxy*log files. For instance:
[logs]$ tail SVNProxyServer-prefs.log 1121854527179 org.nirala.communica......neNet.AsyncConnector makeConnection [logs]$ ../bin/logtimefix SVNProxyServer-prefs.log Crunching SVNProxyServer-prefs.log Wrote ./time-SVNProxyServer-prefs.log [logs]$ tail time-SVNProxyServer-prefs.log Wed Jul 20 03:15:27 -0700 2005 org.nirala.comm.....yncConnector makeConnectionThe UNIX epoch date 1121854527179 was converted to Wed Jul 20 03:15:27 -0700 2005 PST by
logtimefix.
The utility can also be used to report any uncommitted but submitted Subversion transactions. Just pass the -match option and a list of Subversion transaction ids will be printed on the stdout.
commitrate utility. For instance :
[svn-replicator]$ bin/commitrate logs/ProxyServer-prefs.log.0 Crunching logs/ProxyServer-prefs.log.0 Writing output to logs/commits-ProxyServer-prefs.log.0 [svn-replicator]$ tail logs/commits-ProxyServer-prefs.log.0 SVN TxnId, C-S ms, C-S, C-E ms, C-E .......... svn-proposal-d349fd19-fa41-11d9-b957-0011110ee979_50559, 5080, 5 s,11, 11 ms svn-proposal-d349fd19-fa41-11d9-b957-0011110ee979_50562, 5742, 5 s,6, 6 ms svn-proposal-d349fd19-fa41-11d9-b957-0011110ee979_50564, 5788, 5 s,8, 8 ms STATS: Min txn commit time : 3 s Max txn commit time : 11 s Last Txn Submitted at - Thu Jul 21 18:24:00 -0700 2005 Last Txn Committed at - Thu Jul 21 18:24:00 -0700 2005 1294 commits in 11 min, 110 commits/minThe summary statistics are reported at the end of
commits-Proxy* file. Prior to the summary, actual list of committed Subversion transactions with various timing parameters is generated. The column format is - Subversion transaction id, net time to commit after submit in milliseconds and seconds, net time to commit after the transaction starts executing in milliseconds and seconds. The C-S number indicates time consumed by Replicator and Subversion server. The C-E number indicates the time consumed by the backend Subversion server primarily.
WANdisco Subversion Replicator contains a built-in distributed file transfer protocol (DFTP). This is used
to reliably transfer Subversion files and directories between remote replicators.
DFTP optimizes the network bandwidth by streaming the files, using OS level
system calls like sendfile if available. It can automatically open
multiple connections if the connection speed is slow. It can resume
incomplete transfers to deal with network outages or process death.
By default, DFTP uses the same port as Distributed Agreement Engine. Remember - the Distributed Agreement Engine port is multi-protocol so it allows independent TCP connections for multiple protocols. If you want to dedicate a separate network channel for DFTP, you could use a different port than Distributed Agreement Engine. You could then configure a specific QoS in your routers for the DFTP ports.
To setup a DFTP port that is different from the Distributed Agreement Engine port, modify the svn-replicator/config/prefs.xml file as following:
<MemberList>
<Member name="3bfbf219-2918-11d7-80c5-00065be02953">
<Profiles>
<TransportPolicy>AlwaysDConeNet</TransportPolicy>
<Transport>
<DConeNet>
<ListenerIP>localhost</ListenerIP>
<ListenerPort>6020</ListenerPort>
</DConeNet>
<!-- Setup a unique DFTP port /-->
<DConeFTP>
<ListenerIP>localhost</ListenerIP>
<ListenerPort>6021</ListenerPort>
</DConeFTP>
.....
The WANdisco Subversion Replicator allows an administrator to setup a quorum change schedule to move quorum across the replication group. This can give tremendous performance boost for sites separated by multiple timezones. For more details, please see the quorum change section in the Distributed Agreement Engine Administration Guide.
Beyond the basic setup, you may want to tune the network performance, recovery log disk IO performance, change high availability setup. All of these options apply to the Distributed Agreement Engine. Please refer to the Distributed Agreement Engine Administration Guide for further reference.
We recommend gradually increasing the complexity of your deployment scenario, and testing at each stage. Please visit our support web-site for the current deployment checklist in our knowledgebase.
We recommend periodic incremental diffs to ensure
all the remote Subversion repositories are in sync. This could be done periodically
using a cron job for example. But do remember the way WANdisco Subversion Replicator works, it is possible
for two Subversion repositories to appear to be different while updates are happening.
In other words if you take a snapshot while updates are being applied, the snapshots could differ from each other. This does not indicate a failure of WANdisco Subversion Replicator.
So, when making snapshots for comparing, please make sure repositories are not being updated. The easiest way is to -
To do the actual diffs you are welcome to download a fast compare tool from
our support web-site link here. This tool uses MD5 checksums to speed up comparison of Subversion branches at multiple sites on a low-bandwidth, slow network link.
Alternatively you can :
rsync.
diff.
| Copyright © 2005 WANdisco | Sitemap | Privacy Policy | User Agreement | Contact Us |