Re: Hiding data in postgresql

Hi guys,

does ANYONE have any tips on hiding data on a database server? This means
that data is stored in places that is not necessarily picked up in the
schema of the database. I am doing some research on databases and need some
direction.

Any help or direction will be highly appreciated.

Kind regards

Hector

On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.

Let me guess: an attempt at security by obscurity?

I suppose you could always create a couple of columns such that some
function applied over them produced the real result (. You could even
actually store this in an index, so although you could never see the
result directly (except in a dump) queries to get at it might perform
half reasonably.

--
Peter Hunsberger

On 24 May 2010 20:16, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.
> Kind regards
> Hector
>

Why don't role permissions do what you need?

Thom

On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.

Uhhh, initdb -D /dev/null ???

In response to Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>:

> On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> >
> > Hi guys,
> > does ANYONE have any tips on hiding data on a database server? This means
> > that data is stored in places that is not necessarily picked up in the
> > schema of the database. I am doing some research on databases and need some
> > direction.

On a more serious note, I would think the only way to do that would be
to have columns that are misnamed and don't do what you expect.

For example, have an image column that stores image data with your
secret data hidden via steganography. You could do something similar
with some scheme to hide data in text fields or numeric fields, but I
don't know of any specific technique.

Assuming you mean _hide_ and not _secure_. To secure it, encrypt it
(using something like pgcrypto).

But in the schema? I doubt it. PG's data storage is pretty transparent,
you'd probably have to hack the source to pull that off.

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

Wow, this is really an idea I can work with.

I know this is getting really brainstorming, but do you think it is possible
to apply steganography (hiding data in pictures) tactics to the columns of a
database?

Regards
Hector

On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger <
peter(dot)hunsberger(at)gmail(dot)com> wrote:

> On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> >
> > Hi guys,
> > does ANYONE have any tips on hiding data on a database server? This means
> > that data is stored in places that is not necessarily picked up in the
> > schema of the database. I am doing some research on databases and need
> some
> > direction.
> > Any help or direction will be highly appreciated.
>
> Let me guess: an attempt at security by obscurity?
>
> I suppose you could always create a couple of columns such that some
> function applied over them produced the real result (. You could even
> actually store this in an index, so although you could never see the
> result directly (except in a dump) queries to get at it might perform
> half reasonably.
>
> --
> Peter Hunsberger
>

Yes, I mean hide. I am approaching the problem out of the perspective of a
malicious user / hacker.

On Mon, May 24, 2010 at 10:08 PM, Bill Moran <wmoran(at)potentialtech(dot)com>wrote:

> In response to Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>:
>
> > On Mon, May 24, 2010 at 1:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com>
> wrote:
> > >
> > > Hi guys,
> > > does ANYONE have any tips on hiding data on a database server? This
> means
> > > that data is stored in places that is not necessarily picked up in the
> > > schema of the database. I am doing some research on databases and need
> some
> > > direction.
>
> On a more serious note, I would think the only way to do that would be
> to have columns that are misnamed and don't do what you expect.
>
> For example, have an image column that stores image data with your
> secret data hidden via steganography. You could do something similar
> with some scheme to hide data in text fields or numeric fields, but I
> don't know of any specific technique.
>
> Assuming you mean _hide_ and not _secure_. To secure it, encrypt it
> (using something like pgcrypto).
>
> But in the schema? I doubt it. PG's data storage is pretty transparent,
> you'd probably have to hack the source to pull that off.
>
> --
> Bill Moran
> http://www.potentialtech.com
> http://people.collaborativefusion.com/~wmoran/
>

On Mon, May 24, 2010 at 3:10 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> Wow, this is really an idea I can work with.
> I know this is getting really brainstorming, but do you think it is possible
> to apply steganography (hiding data in pictures) tactics to the columns of a
> database?

If by that, you mean a variation on my original reply, then yes, but
you'd need at least one column that did not necessarily have contents
that looked like normal text. This column would work as some form of
offset pointer into the other columns. The result wouldn't
necessarily require much in the way of a reconstruction algorithm, but
if not it would likely also be fairly easy to figure out if someone
stumbled across your scheme.

If you want the truly secure variation on this then use some public
key / private key scheme in conjunction with this (in particular, ECC
could be pretty fast and have a small key size), so that some portion
of this is encrypted. However, if you're doing that, you might as
well just encrypt the data directly...

--
Peter Hunsberger

Dear Peter,

can you elaborate on what you mean by storing 'this' in the index. Are you
referring to the function that is applied over the data?

How would you be able to see the result with a dump?

Thanks, your ideas are really helping...

Regards
Hector

On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger <
peter(dot)hunsberger(at)gmail(dot)com> wrote:

> On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> >
> > Hi guys,
> > does ANYONE have any tips on hiding data on a database server? This means
> > that data is stored in places that is not necessarily picked up in the
> > schema of the database. I am doing some research on databases and need
> some
> > direction.
> > Any help or direction will be highly appreciated.
>
> Let me guess: an attempt at security by obscurity?
>
> I suppose you could always create a couple of columns such that some
> function applied over them produced the real result (. You could even
> actually store this in an index, so although you could never see the
> result directly (except in a dump) queries to get at it might perform
> half reasonably.
>
> --
> Peter Hunsberger
>

On Mon, May 24, 2010 at 3:24 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> Dear Peter,
> can you elaborate on what you mean by storing 'this' in the index. Are you
> referring to the function that is applied over the data?
> How would you be able to see the result with a dump?

Yes, you would the function to create the index, something like:

create index gunk on tbl1 ( hidefunc( a, b, c ) )

The data created in this index might be viewable in a database dump by
simple inspection with something like an editor. How meaningful it
might be is another question...

--
Peter Hunsberger

Acutally, that's probably the best way to this. The key to effective
steganography is having a large amount of data to store a small amount
of data. So, if you don't mind having a db that's 10 to 100 times
bigger than it has to be to store the original data it should work.

On Mon, May 24, 2010 at 2:10 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> Wow, this is really an idea I can work with.
> I know this is getting really brainstorming, but do you think it is possible
> to apply steganography (hiding data in pictures) tactics to the columns of a
> database?
> Regards
> Hector
>
>
>
> On Mon, May 24, 2010 at 9:30 PM, Peter Hunsberger
> <peter(dot)hunsberger(at)gmail(dot)com> wrote:
>>
>> On Mon, May 24, 2010 at 2:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>> >
>> > Hi guys,
>> > does ANYONE have any tips on hiding data on a database server? This
>> > means
>> > that data is stored in places that is not necessarily picked up in the
>> > schema of the database. I am doing some research on databases and need
>> > some
>> > direction.
>> > Any help or direction will be highly appreciated.
>>
>> Let me guess: an attempt at security by obscurity?
>>
>> I suppose you could always create a couple of columns such that some
>> function applied over them produced the real result (.  You could even
>> actually store this in an index, so although you could never see the
>> result directly (except in a dump) queries to get at it might perform
>> half reasonably.
>>
>> --
>> Peter Hunsberger
>
>

--
When fascism comes to America, it will be intolerance sold as diversity.

On Mon, May 24, 2010 at 3:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>
> Hi guys,
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
> Any help or direction will be highly appreciated.

First question: Have you considered 1. encrypting data when you put it
in the database and 2. decrypting it when you pull it out?

Let me humbly state that the #1 problem that beginners face with
security and encryption is focusing too much on the mechanics and not
enough on the 'big picture' issues:

*) What data is to remain secret?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?

Answers to those questions should get you more helpful answers.
Postgres has a lot of features to hide data, some obvious (pgcrypto,
grant/revoke) and some not so obvious (revoking permissions from
pg_proc). Judging from your question you may be interested in some
extra-special techniques...please be more specific!

merlin

No, I have not considered encrypting or decrypting data. The reason for this
is that I am trying to *secure a database* by thinking like a *malicious
user / criminal*. I want to hide (for example) fraudulent data on a database
where it is not easily seen by others and then build a tool to detect this
hidden data.

On your questions:

*) What data is to remain secret?
*) Who is allowed to see the secret data?
*) When do they see it?
*) What sacrifices are you willing to make to keep the data secret?
*) Where are you going to store the key?

the answers:

- fraudulent data / or data that needs to be hidden.
- only the malicious user - and hopefully later a detection mechanism
that I aim to build.
- I don't really have a preference on when they can see the data, but
maybe when you export a dump.
- The main purpose of hiding the data is that the normal users of the
database will not easily find the hidden data. If this criteria is met, then
any other sacrifices can be made.
- Still need to figure that one out.

Any good brainstorming ideas will help!

On Mon, May 24, 2010 at 11:04 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:

> On Mon, May 24, 2010 at 3:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> >
> > Hi guys,
> > does ANYONE have any tips on hiding data on a database server? This means
> > that data is stored in places that is not necessarily picked up in the
> > schema of the database. I am doing some research on databases and need
> some
> > direction.
> > Any help or direction will be highly appreciated.
>
> First question: Have you considered 1. encrypting data when you put it
> in the database and 2. decrypting it when you pull it out?
>
> Let me humbly state that the #1 problem that beginners face with
> security and encryption is focusing too much on the mechanics and not
> enough on the 'big picture' issues:
>
> *) What data is to remain secret?
> *) Who is allowed to see the secret data?
> *) When do they see it?
> *) What sacrifices are you willing to make to keep the data secret?
> *) Where are you going to store the key?
>
> Answers to those questions should get you more helpful answers.
> Postgres has a lot of features to hide data, some obvious (pgcrypto,
> grant/revoke) and some not so obvious (revoking permissions from
> pg_proc). Judging from your question you may be interested in some
> extra-special techniques...please be more specific!
>
> merlin
>

On Mon, May 24, 2010 at 05:04:10PM -0400, Merlin Moncure wrote:
> Let me humbly state that the #1 problem that beginners face with
> security and encryption is focusing too much on the mechanics and not
> enough on the 'big picture' issues:

One more that OP seems to be avoiding is why would anybody want to do
this anyway? There are plenty of places that will happily host data for
you--most email sites give you many gigabytes of storage these days.
Seems to be a solution in search of a problem to me.

I think what the OP may be asking is about the presence of "covert
channels". There are plenty of these in PG, an attacker can do the
obvious things like disguising data inside other data (steganography) or
more subtle things like tuple order on disk, transaction orderings, or
interactions between running queries (i.e. causing one to pause for a
few milliseconds by reading/locking a table).

Covert channels seem to be a fundamental fact of nature. As far as I
know, though I'm not aware of any papers directly on the subject, it's
*always* possible to design a new attack by exploiting the physical
implementation of something. Hence any specific tool you design to look
for any specific attack can always be avoided in an infinite number of
ways, generally negating its purpose. You have to be much more specific
in your requirements before useful analysis can be done.

What can be done is to reduce the bandwidth of a specific covert
channel, and beyond some threshold it *may* be possible to say that "no
useful data can be transmitted", but that's about it. If somebody just
wants to leak a password/private key a surprisingly few number of bits
will go a long way.

--
Sam http://samason.me.uk/

Hi guys,

thank you for your replies yesterday on this topic. I have one more question
though:

Does someone have any ideas how I can hide data without the meta data
noticing? To explain further, I would like to save some collection of data
where the meta-data does not see it. I am trying to do some security through
obscurity. It is for research purposes.

Maybe to save populate a table with 1000 rows, but the meta-data only knows
of about 500 of them? Only on an export of a dump can you find the data
again.

Kind regards
Hector

On Mon, May 24, 2010 at 9:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:

>
> Hi guys,
>
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
>
> Any help or direction will be highly appreciated.
>
> Kind regards
>
> Hector
>
>

Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:

> thank you for your replies yesterday on this topic. I have one more question
> though:

> Does someone have any ideas how I can hide data without the meta data
> noticing? To explain further, I would like to save some collection of data
> where the meta-data does not see it. I am trying to do some security through
> obscurity. It is for research purposes.

> Maybe to save populate a table with 1000 rows, but the meta-data only knows
> of about 500 of them? Only on an export of a dump can you find the data
> again.
> [...]

Before delving deeper into this, you should get your termi-
nology straight: What do you mean by "meta-data"? What do
you mean by "export of a dump"? What do you mean by "without
the meta data noticing"?

Tim

On 5/24/2010 3:18 PM, Hector Beyers wrote:
> Yes, I mean hide. I am approaching the problem out of the perspective
> of a malicious user / hacker.
>
> **snip***

First hiding data is not a solution to secure or block access to
information. This only slows people down it does not stop them, never
underestimate users with access to the data

It would be helpful to explain the type of data that needs to be
hidden/secured

Example of failed attempts to hide data is to look at the numerous
mistakes in securing credit card data at many Companies.

In almost every case that i have read the programmers just tried to hide
the data or limit access instead of doing Public Key Private Key
encryption methodology . I know of several big name apps that still
store credit card data where the end users can reverse the encryption
meaning if the key becomes unsecured any the data is visible that is
encrypted.

I have seen where the data is only encrypted inside the database so the
information is transmitted in the clear to the client as the database
decrypted the data on the fly . What is the point??

Trying to hide information is waste of time and energy look into
encryption.

All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored.

CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately.
Thank you.

Hi guys,

(I tried the question in another forum first)

Does someone have any ideas how I can hide data without the meta data
noticing? To explain further, I would like to save some collection of data
where the meta-data does not see it. I am trying to do some security through
obscurity. It is for research purposes.

For example, populate a table with 1000 rows, but the meta-data only knows
of about 500 of them? Only on an export of a dump can you find all the data
again. Or maybe to make a hidden duplicate schema that can point to the
hidden data?

Does someone have any good ideas on how to achieve this or something
similar?

Kind regards
Hector

On Mon, May 24, 2010 at 9:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:

>
> Hi guys,
>
> does ANYONE have any tips on hiding data on a database server? This means
> that data is stored in places that is not necessarily picked up in the
> schema of the database. I am doing some research on databases and need some
> direction.
>
> Any help or direction will be highly appreciated.
>
> Kind regards
>
> Hector
>
>

On Tue, May 25, 2010 at 11:30 AM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
> Maybe to save populate a table with 1000 rows, but the meta-data only knows
> of about 500 of them? Only on an export of a dump can you find the data
> again.

Well, here's your flaw: If you expect the data dumper to be able to
export the rows, then the system must know that the rows/tables are
there, and thus cannot be "hidden" for any definition of hidden. To
find them, you just do the work that the dumper would do.

Ie, you seem to want to be able to write files that postgres doesn't
know are there, and that I don't believe you can do.

On Tue, May 25, 2010 at 3:39 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>
> Hi guys,
> (I tried the question in another forum first)
> Does someone have any ideas how I can hide data without the meta data
> noticing? To explain further, I would like to save some collection of data
> where the meta-data does not see it. I am trying to do some security through
> obscurity. It is for research purposes.
> For example, populate a table with 1000 rows, but the meta-data only knows
> of about 500 of them? Only on an export of a dump can you find all the data
> again. Or maybe to make a hidden duplicate schema that can point to the
> hidden data?
> Does someone have any good ideas on how to achieve this or something
> similar?
> Kind regards
> Hector
>
>
> On Mon, May 24, 2010 at 9:16 PM, Hector Beyers <hqbeyers(at)gmail(dot)com> wrote:
>>
>> Hi guys,
>> does ANYONE have any tips on hiding data on a database server? This means
>> that data is stored in places that is not necessarily picked up in the
>> schema of the database. I am doing some research on databases and need some
>> direction.
>> Any help or direction will be highly appreciated.
>> Kind regards
>> Hector

Not sure if this helpful, but be sure to know about views, which can
be used to filter out rows of a table. Example:

CREATE TABLE foo (name TEXT, visible BOOL);
INSERT INTO foo VALUES ('two', true);
INSERT INTO foo VALUES ('three', true);
INSERT INTO foo VALUES ('four', false);
INSERT INTO foo VALUES ('five', true);
INSERT INTO foo VALUES ('six', false);
INSERT INTO foo VALUES ('seven', true);
INSERT INTO foo VALUES ('eight', false);
INSERT INTO foo VALUES ('nine', false);
INSERT INTO foo VALUES ('ten', false);
INSERT INTO foo VALUES ('eleven', true);

CREATE VIEW foo_view AS SELECT foo.name FROM foo WHERE visible=true;

=> SELECT * FROM foo;
name | visible
--------+---------
two | t
three | t
four | f
five | t
six | f
seven | t
eight | f
nine | f
ten | f
eleven | t
(10 rows)

=> SELECT * FROM foo_view;
name
--------
two
three
five
seven
eleven
(5 rows)

Note that views are SELECT-only, but you can use CREATE RULE to
simulate an updatable view.

You may also want to read about Veil:
http://veil.projects.postgresql.org/curdocs/main.html

On 5/25/2010 2:58 AM, Hector Beyers wrote:
>
> No, I have not considered encrypting or decrypting data. The reason
> for this is that I am trying to /secure a database/ by thinking like a
> /malicious user / criminal/. I want to hide (for example) fraudulent
> data on a database where it is not easily seen by others and then
> build a tool to detect this hidden data.
>
> On your questions:
>
> *) What data is to remain secret?
> *) Who is allowed to see the secret data?
> *) When do they see it?
> *) What sacrifices are you willing to make to keep the data secret?
> *) Where are you going to store the key?
>
> the answers:
>
> * fraudulent data / or data that needs to be hidden.
> * only the malicious user - and hopefully later a detection
> mechanism that I aim to build.
> * I don't really have a preference on when they can see the data,
> but maybe when you export a dump.
> * The main purpose of hiding the data is that the normal users of
> the database will not easily find the hidden data. If this
> criteria is met, then any other sacrifices can be made.
> * Still need to figure that one out.
>
>
> Any good brainstorming ideas will help!

Missed this bit prior to first responds.

I think some of the assumptions here are flawed.

If hacker actually got into a database why would they do this??? what
is being accomplished??? why would anyone want to do this???

Again it would make allot more sense if a hacker stored data in plain
site. Create tables that look like real tables following the same
naming schema or use already existing tables like logs, Modify the
tables adding columns to store data. Then create/update records
encrypting the contents, this would protect the contents from ever being
read by anyone except by the creator.

Think this line through how long would a Hacker go unnoticed if they
used the already existing tables adding in columns or take over stale
records like old customers that are no longer active. Then use the text
fields to store data. The hacker could create normal user account to
access those records throwing up no red flags. How many people review
table structures or update to already existing records.

The current crop of hackers are not hexeditor high-school wannabe's.
Hackers want to go unnoticed for as long as they can so that means doing
nothing out of ordinary that throws up red flags.

Just read up on the investigations on stolen credit cards. Or fake ATMS
that's been installed at malls. The hackers/thieves figured out how to
go unnoticed for long periods of time by appearing normal.

Second assumption is the hacker actual got a admin/root level access
to be able to do these kind of things. This means security upfront was
lacks which point there are far bigger problems than hidden data.

Far better way to secure is not trying think what they can do once they
get access, but stop them getting in to start with. If anyone gets
this high level of access protecting from or figuring out if they have
hidden data is immaterial to the problems someone has.

All legitimate Magwerks Corporation quotations are sent in a .PDF file attachment with a unique ID number generated by our proprietary quotation system. Quotations received via any other form of communication will not be honored.

CONFIDENTIALITY NOTICE: This e-mail, including attachments, may contain legally privileged, confidential or other information proprietary to Magwerks Corporation and is intended solely for the use of the individual to whom it addresses. If the reader of this e-mail is not the intended recipient or authorized agent, the reader is hereby notified that any unauthorized viewing, dissemination, distribution or copying of this e-mail is strictly prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and destroy all occurrences of this e-mail immediately.
Thank you.

Hector,

* Hector Beyers (hqbeyers(at)gmail(dot)com) wrote:
> Does someone have any ideas how I can hide data without the meta data
> noticing? To explain further, I would like to save some collection of data
> where the meta-data does not see it. I am trying to do some security through
> obscurity. It is for research purposes.

This explanation doesn't actually explain anything, near as I can tell.
Perhaps if you would share what your actual problem is, we could
recommend a solution.

Thanks,

Stephen