Discussion:
Beginner question on db restore
(too old to reply)
Robert Han
2011-10-28 19:17:15 UTC
Permalink
Hi all,

Pardon for the beginner question. I recently did a restore from a back-up of
an eXist from another PC user. After restoring, my existing username and
password won't work, do I need to use the ID/Pasword from that instance of
eXist?

Thanks!

Rob
Jens Østergaard Petersen
2011-11-07 09:02:41 UTC
Permalink
Hi Rob,

This is not really a straight answer, since I have questions about this as well, but here is my take on your question.

When you do a restore, importing a zip file using the java admin client, what is inside the backup will overwrite what is in identical positions on the live system. What is not in identical positions will be left as it is.

If you have a full backup of /db, this backup will include /db/system/users.xml and /db/security/exist/accounts. If you restore such a backup, the information on users and passwords on your live system will be overwritten by the information on users and passwords on the system backed up.

If you are restoring another user's backup, you will then have to authenticate with the other user's credentials when logging in. - I was myself trapped by this, since I unthinkingly imported the backup posted to the list by Alem Areki on the day of your posting - and was locked out of my database as a result (but I had a full backup of my installation, so everything is cool).

On <http://exist-db.org/backup.html#d1864e846>, the restore process is described in (what I consider) a roundabout and unclear way:

"Restoring from a backup (or parts of it) does not mean that the existing data in the current database instance will be deleted entirely. The restore process will upload the collections and documents contained in the backup. Collections and documents which exist in the database but are not part of the backup will not be modified."

What I _believe_ you should do (but don't take my word for it!) is to unzip the backup, remove /db/system/users.xml and /db/security/. You should then remove the information relating to the document "users.xml" and the "security" collection from the /db/system/__contents__.xml file. Finally, you should zip the db directory again and do the restore. You can then log in with your own credentials.

You don't have to zip; if you don't, follow the instructions on <http://exist-db.org/backup.html#d1864e900>, choosing "__content.xml__ files" in the File Format drop-down menu and selecting the uppermost __content.xml__ file.

The last thing I want to do is to give security-related advice, so until someone more knowledgable OKs this, you should not do anything!

Best,

Jens
Post by Robert Han
Hi all,
Pardon for the beginner question. I recently did a restore from a back-up of an eXist from another PC user. After restoring, my existing username and password won't work, do I need to use the ID/Pasword from that instance of eXist?
Thanks!
Rob
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
Jens Østergaard Petersen
2011-11-18 09:48:51 UTC
Permalink
Hi,

I am reposting this from more than a week ago. Could someone tell me if this is an acceptable way to restore a database dump you, for instance, do not know the admin password for?

Cheers,

Jens


Subject: Re: [Exist-open] Beginner question on db restore

Hi Rob,

This is not really a straight answer, since I have questions about this as well, but here is my take on your question.

When you do a restore, importing a zip file using the java admin client, what is inside the backup will overwrite what is in identical positions on the live system. What is not in identical positions will be left as it is.

If you have a full backup of /db, this backup will include /db/system/users.xml and /db/security/exist/accounts. If you restore such a backup, the information on users and passwords on your live system will be overwritten by the information on users and passwords on the system backed up.

If you are restoring another user's backup, you will then have to authenticate with the other user's credentials when logging in. - I was myself trapped by this, since I unthinkingly imported the backup posted to the list by Alem Areki on the day of your posting - and was locked out of my database as a result (but I had a full backup of my installation, so everything is cool).

On <http://exist-db.org/backup.html#d1864e846>, the restore process is described in (what I consider) a roundabout and unclear way:

"Restoring from a backup (or parts of it) does not mean that the existing data in the current database instance will be deleted entirely. The restore process will upload the collections and documents contained in the backup. Collections and documents which exist in the database but are not part of the backup will not be modified."

What I _believe_ you should do (but don't take my word for it!) is to unzip the backup, remove /db/system/users.xml and /db/security/. You should then remove the information relating to the document "users.xml" and the "security" collection from the /db/system/__contents__.xml file. Finally, you should zip the db directory again and do the restore. You can then log in with your own credentials.

You don't have to zip; if you don't, follow the instructions on <http://exist-db.org/backup.html#d1864e900>, choosing "__content.xml__ files" in the File Format drop-down menu and selecting the uppermost __content.xml__ file.

The last thing I want to do is to give security-related advice, so until someone more knowledgable OKs this, you should not do anything!

Best,

Jens
Post by Robert Han
Hi all,
Pardon for the beginner question. I recently did a restore from a back-up of an eXist from another PC user. After restoring, my existing username and password won't work, do I need to use the ID/Pasword from that instance of eXist?
Thanks!
Rob
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
Hungerburg
2011-11-18 10:54:32 UTC
Permalink
Post by Jens Østergaard Petersen
Hi,
I am reposting this from more than a week ago. Could someone tell me if this is an acceptable way to restore a database dump you, for instance, do not know the admin password for?
[...]

Looks practicable from my /users/ point of view. Maybe you could also
just replace the admin passwd in the backup with a known one or none at
all, so any referenced accounts in the backup can be resolved later.
Depending on how it is crypted this might work. Or even simpler:

Does restoring such a backup prompt you to relogin the webstart admin
client, when the password has changed? If the running admin session
remains valid, you could even just do the backup and set the admin
password as an immediate next step.
--
peter
Jens Østergaard Petersen
2011-11-21 09:23:52 UTC
Permalink
Hi Peter,

Thank you for your thoughts.
Post by Hungerburg
Post by Jens Østergaard Petersen
Hi,
I am reposting this from more than a week ago. Could someone tell me if this is an acceptable way to restore a database dump you, for instance, do not know the admin password for?
[...]
Looks practicable from my /users/ point of view. Maybe you could also
just replace the admin passwd in the backup with a known one or none at
all, so any referenced accounts in the backup can be resolved later.
Depending on how it is crypted this might work.
Well, the info looks like this in 1.5.0:

<account xmlns="http://exist-db.org/Configuration" id="1048574"><password>{MD5}WhBei51A4TKXgNYuoiZdig==</password><digestPassword>test1</digestPassword>

It is true, as you say, that if I copy the corresponding string from another admin password, I can change passwords in the database I restore. However, deleting the files strikes me as safer.

What I find a little worrying is that the password appears in the clear in 1.5.0: in 1.4.1, all passwords are encrypted.

In 1.4.1, the same password ("test1") is rendered as

<user name="admin" uid="1" password="{MD5}WhBei51A4TKXgNYuoiZdig==" digest-password="4596428fed747447731bdd7722a08031" home="">

The 1.4.1 behaviour is obviously the correct one - that is, after all, what "digest password" means ….

I guess there is a reason why the password has to be "digested" twice …
Post by Hungerburg
Does restoring such a backup prompt you to relogin the webstart admin
client, when the password has changed? If the running admin session
remains valid, you could even just do the backup and set the admin
password as an immediate next step.
Yes, if I know the password of the installation I am restoring, what you say is correct. I log in with the base installation's admin password and enter the backup's password prior to restore. At this point, if I logged in again, I would have to enter the admin password used for the backup, but if I do not, I can e.g. change the password of the installation to something else. This appears to be OK, security-wise.

However, if I give the wrong password, that is, a password that is not the password of the backup, the process aborts - but not before it has overwritten the security files. This means that if I don't know this password, I am out of luck: my database becomes inaccessible.

If I try to do a restore when logged into the admin client as a user without admin status, the process aborts before anything is overwritten, so here everything is cool.

Jens
Post by Hungerburg
--
peter
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
Martin Holmes
2011-11-21 21:46:35 UTC
Permalink
This issue of the password being in clear text bothers me greatly; it
means I have to take great care about where I store my backups, which
wasn't something I'd ever worried about before.

Is there a bug report in for this? If not, should we submit one? It
seems urgent to me.

Cheers,
Martin
Post by Jens Østergaard Petersen
Hi Peter,
Thank you for your thoughts.
Post by Hungerburg
Post by Jens Østergaard Petersen
Hi,
I am reposting this from more than a week ago. Could someone tell me if this is an acceptable way to restore a database dump you, for instance, do not know the admin password for?
[...]
Looks practicable from my /users/ point of view. Maybe you could also
just replace the admin passwd in the backup with a known one or none at
all, so any referenced accounts in the backup can be resolved later.
Depending on how it is crypted this might work.
<account xmlns="http://exist-db.org/Configuration" id="1048574"><password>{MD5}WhBei51A4TKXgNYuoiZdig==</password><digestPassword>test1</digestPassword>
It is true, as you say, that if I copy the corresponding string from another admin password, I can change passwords in the database I restore. However, deleting the files strikes me as safer.
What I find a little worrying is that the password appears in the clear in 1.5.0: in 1.4.1, all passwords are encrypted.
In 1.4.1, the same password ("test1") is rendered as
<user name="admin" uid="1" password="{MD5}WhBei51A4TKXgNYuoiZdig==" digest-password="4596428fed747447731bdd7722a08031" home="">
The 1.4.1 behaviour is obviously the correct one - that is, after all, what "digest password" means ….
I guess there is a reason why the password has to be "digested" twice …
Post by Hungerburg
Does restoring such a backup prompt you to relogin the webstart admin
client, when the password has changed? If the running admin session
remains valid, you could even just do the backup and set the admin
password as an immediate next step.
Yes, if I know the password of the installation I am restoring, what you say is correct. I log in with the base installation's admin password and enter the backup's password prior to restore. At this point, if I logged in again, I would have to enter the admin password used for the backup, but if I do not, I can e.g. change the password of the installation to something else. This appears to be OK, security-wise.
However, if I give the wrong password, that is, a password that is not the password of the backup, the process aborts - but not before it has overwritten the security files. This means that if I don't know this password, I am out of luck: my database becomes inaccessible.
If I try to do a restore when logged into the admin client as a user without admin status, the process aborts before anything is overwritten, so here everything is cool.
Jens
Post by Hungerburg
--
peter
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
Wolfgang Meier
2011-11-21 22:31:02 UTC
Permalink
Post by Martin Holmes
This issue of the password being in clear text bothers me greatly; it
means I have to take great care about where I store my backups, which
wasn't something I'd ever worried about before.
eXist only keeps and compares the digested password internally. The
clear text password should not be known to eXist, so I'm really
surprised it ends up in the files now. Dmitryi, any idea where this is
coming from?

Wolfgang
Dmitriy Shabanov
2011-11-22 04:38:12 UTC
Permalink
Post by Wolfgang Meier
Post by Martin Holmes
This issue of the password being in clear text bothers me greatly; it
means I have to take great care about where I store my backups, which
wasn't something I'd ever worried about before.
eXist only keeps and compares the digested password internally. The
clear text password should not be known to eXist, so I'm really
surprised it ends up in the files now. Dmitryi, any idea where this is
coming from?
Most surprising is 'digestPassword', it shouldn't be there at all. What is
the steps to reproduce it?

Note: to break MD5 I need few ms, oops .... back should be in safe/secure
place any way!!!

I do waiting Adam to finish with redesign to submit switch to SHA as
minimum (make it configurable).
--
Dmitriy Shabanov
Wolfgang Meier
2011-11-22 09:48:37 UTC
Permalink
Post by Dmitriy Shabanov
Most surprising is 'digestPassword', it shouldn't be there at all. What is
the steps to reproduce it?
Set a password for admin and create a backup without restarting eXist.
If you restart eXist, digestPassword disappears again.
Post by Dmitriy Shabanov
Note: to break MD5 I need few ms, oops .... back should be in safe/secure
place any way!!!
I originally introduced the MD5 digest to support WebDAV. But if it
doesn't cause any issues, replacing it with SHA on new installs would
be great. But please remember that users still need to be able to
restore a 3 years old backup ;-)

Wolfgang
Dmitriy Shabanov
2011-11-22 12:38:04 UTC
Permalink
Post by Wolfgang Meier
Post by Dmitriy Shabanov
Most surprising is 'digestPassword', it shouldn't be there at all. What
is
Post by Dmitriy Shabanov
the steps to reproduce it?
Set a password for admin and create a backup without restarting eXist.
If you restart eXist, digestPassword disappears again.
Ok, I'll check.
Post by Wolfgang Meier
Post by Dmitriy Shabanov
Note: to break MD5 I need few ms, oops .... back should be in safe/secure
place any way!!!
I originally introduced the MD5 digest to support WebDAV. But if it
doesn't cause any issues, replacing it with SHA on new installs would
be great. But please remember that users still need to be able to
restore a 3 years old backup ;-)
Yes, that's why {MD5} prefix there. That information can be used to choose
algorithm.
--
Dmitriy Shabanov
Hungerburg
2011-11-22 13:26:57 UTC
Permalink
Post by Dmitriy Shabanov
Post by Wolfgang Meier
Set a password for admin and create a backup without restarting eXist.
If you restart eXist, digestPassword disappears again.
Ok, I'll check.
For me this would not make the plaintext password vanish from admin.xml
in /etc/... though I did set the password to the same one and not a
different one, so maybe...
--
peter
Martin Holmes
2011-11-21 21:46:35 UTC
Permalink
This issue of the password being in clear text bothers me greatly; it
means I have to take great care about where I store my backups, which
wasn't something I'd ever worried about before.

Is there a bug report in for this? If not, should we submit one? It
seems urgent to me.

Cheers,
Martin
Post by Jens Østergaard Petersen
Hi Peter,
Thank you for your thoughts.
Post by Hungerburg
Post by Jens Østergaard Petersen
Hi,
I am reposting this from more than a week ago. Could someone tell me if this is an acceptable way to restore a database dump you, for instance, do not know the admin password for?
[...]
Looks practicable from my /users/ point of view. Maybe you could also
just replace the admin passwd in the backup with a known one or none at
all, so any referenced accounts in the backup can be resolved later.
Depending on how it is crypted this might work.
<account xmlns="http://exist-db.org/Configuration" id="1048574"><password>{MD5}WhBei51A4TKXgNYuoiZdig==</password><digestPassword>test1</digestPassword>
It is true, as you say, that if I copy the corresponding string from another admin password, I can change passwords in the database I restore. However, deleting the files strikes me as safer.
What I find a little worrying is that the password appears in the clear in 1.5.0: in 1.4.1, all passwords are encrypted.
In 1.4.1, the same password ("test1") is rendered as
<user name="admin" uid="1" password="{MD5}WhBei51A4TKXgNYuoiZdig==" digest-password="4596428fed747447731bdd7722a08031" home="">
The 1.4.1 behaviour is obviously the correct one - that is, after all, what "digest password" means ….
I guess there is a reason why the password has to be "digested" twice …
Post by Hungerburg
Does restoring such a backup prompt you to relogin the webstart admin
client, when the password has changed? If the running admin session
remains valid, you could even just do the backup and set the admin
password as an immediate next step.
Yes, if I know the password of the installation I am restoring, what you say is correct. I log in with the base installation's admin password and enter the backup's password prior to restore. At this point, if I logged in again, I would have to enter the admin password used for the backup, but if I do not, I can e.g. change the password of the installation to something else. This appears to be OK, security-wise.
However, if I give the wrong password, that is, a password that is not the password of the backup, the process aborts - but not before it has overwritten the security files. This means that if I don't know this password, I am out of luck: my database becomes inaccessible.
If I try to do a restore when logged into the admin client as a user without admin status, the process aborts before anything is overwritten, so here everything is cool.
Jens
Post by Hungerburg
--
peter
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
Dmitriy Shabanov
2011-11-22 04:43:57 UTC
Permalink
Post by Jens Østergaard Petersen
<account xmlns="http://exist-db.org/Configuration"
id="1048574"><password>{MD5}WhBei51A4TKXgNYuoiZdig==</password><digestPassword>test1</digestPassword>
What is value of 'password-encoding' at conf.xml?
--
Dmitriy Shabanov
Jens Østergaard Petersen
2011-11-22 07:50:03 UTC
Permalink
Post by Jens Østergaard Petersen
<account xmlns="http://exist-db.org/Configuration" id="1048574"><password>{MD5}WhBei51A4TKXgNYuoiZdig==</password><digestPassword>test1</digestPassword>
What is value of 'password-encoding' at conf.xml?
EXIST_HOME/conf.xml has nothing on passwords or encoding (<http://exist.svn.sourceforge.net/viewvc/exist/trunk/eXist/conf.xml.tmpl?revision=15470&pathrev=15540>).

In EXIST_HOME/webapp/configuration.xml, line 1170, 'password-encoding' occurs; this information once occurred in EXIST_HOME/conf.xml - you moved it with revision 14136.

Otherwise 'password-encoding' occurs only in EXIST_HOME/src/org/exist/util/Configuration.java. A change were made on April 19 which looks like it could have an effect on password encoding: <http://exist.svn.sourceforge.net/viewvc/exist/trunk/eXist/src/org/exist/util/Configuration.java?r1=13941&r2=14245&pathrev=14245>.

I made a backup of /db/system on April 30 (where I may not have been up-to-date) and here the <digestPassword> is in MD5 digest form. It is identical to <password>.

Jens
Post by Jens Østergaard Petersen
--
Dmitriy Shabanov
Adam Retter
2011-11-21 13:45:38 UTC
Permalink
Jens,

You approach seems acceptable to me.

I think there is a larger question here - Should restoring a backup
overwrite the current admin password, or in fact any password of a
user that exists in both the db and the backup to be restored?

My feeling is that the default behaviour should NOT overwrite the
password of any existing users. This could of course be made
configurable.
Post by Jens Østergaard Petersen
Hi,
I am reposting this from more than a week ago. Could someone tell me if this
is an acceptable way to restore a database dump you, for instance, do not
know the admin password for?
Cheers,
Jens
Subject: Re: [Exist-open] Beginner question on db restore
Hi Rob,
This is not really a straight answer, since I have questions about this as
well, but here is my take on your question.
When you do a restore, importing a zip file using the java admin client,
what is inside the backup will overwrite what is in identical positions on
the live system. What is not in identical positions will be left as it is.
If you have a full backup of /db, this backup will include
/db/system/users.xml and /db/security/exist/accounts. If you restore such a
backup, the information on users and passwords on your live system will be
overwritten by the information on users and passwords on the system backed
up.
If you are restoring another user's backup, you will then have to
authenticate with the other user's credentials when logging in. - I was
myself trapped by this, since I unthinkingly imported the backup posted to
the list by Alem Areki on the day of your posting - and was locked out of my
database as a result (but I had a full backup of my installation, so
everything is cool).
On <http://exist-db.org/backup.html#d1864e846>, the restore process is
"Restoring from a backup (or parts of it) does not mean that the existing
data in the current database instance will be deleted entirely. The restore
process will upload the collections and documents contained in the backup.
Collections and documents which exist in the database but are not part of
the backup will not be modified."
What I _believe_ you should do (but don't take my word for it!) is to unzip
the backup, remove /db/system/users.xml and /db/security/. You should then
remove the information relating to the document "users.xml" and the
"security" collection from the /db/system/__contents__.xml file. Finally,
you should zip the db directory again and do the restore. You can then log
in with your own credentials.
You don't have to zip; if you don't, follow the instructions on
<http://exist-db.org/backup.html#d1864e900>, choosing "__content.xml__
files" in the File Format drop-down menu and selecting the uppermost
__content.xml__ file.
The last thing I want to do is to give security-related advice, so until
someone more knowledgable OKs this, you should not do anything!
Best,
Jens
Post by Robert Han
Hi all,
Pardon for the beginner question. I recently did a restore from a back-up
of an eXist from another PC user. After restoring, my existing username
and password won't work, do I need to use the ID/Pasword from that
instance of eXist?
Thanks!
Rob
------------------------------------------------------------------------------
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure
contains a definitive record of customers, application performance,
security threats, fraudulent activity, and more. Splunk takes this
data and makes sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-novd2d
_______________________________________________
Exist-open mailing list
https://lists.sourceforge.net/lists/listinfo/exist-open
--
Adam Retter

eXist Developer
{ United Kingdom }
***@exist-db.org
irc://irc.freenode.net/existdb
Continue reading on narkive:
Loading...