Processing the Return...

[jeremyevans]

Google Group Post: https://groups.google.com/g/ruby-forme/c/QSIBl0Ijm4c
Google Group Date: Mon, 19 May 2014 23:12:25 -0700
Google Group Sender: [email protected]

I guess I’m still not sure what do do with what comes back.

I’m getting back

order={"status"=>"open", "saledate"=>"2014 May 19 20:02", "invoicedate"=>"", "shippeddate"=>"", "emaileddate"=>"", "shippingcost"=>"", "container_id"=>"a9b8faa6-5ea9-11e3-8601-003048686938", "source_id"=>"9a4a201e-0a7b-11e2-a873-003048686938", "note"=>"", "items_attributes"=>{"0"=>{"id"=>"3b22c47a-dfcb-11e3-925b-003048686938", "quantity"=>"1", "sku_id"=>"pgrpt", "name"=>"PennyGems Relative P/T Sheet", "price"=>"9.75", "shipping"=>"", "discount"=>"", "salestax"=>""}}}

and . . . hey, wait . . . there’s no ID for the order itself. Surely I don’t have to explicity add the primary key? Oh, no, because even if I do, it’s still not included in the form. f.input(:id) does not create an tag.

Well, then, no wonder I got this error:

Sequel::Error at /orders/savechanges
no matching associated object with given primary key (association: items, pk: ["3b22c47a-dfcb-11e3-925b-003048686938”]

Because the item with that PK belongs to Order 29823, and I don’t know how to update that order with these data…

Other than that, this kinda worked….

  order = Order.new(request.params["order"])
  order.save

Although I suspect I really need to do a .create_or_update

[jeremyevans]

Google Group Date: Tue, 20 May 2014 08:23:23 -0700
Google Group Sender: [email protected]

On Monday, May 19, 2014 11:12:25 PM UTC-7, Dave Howell wrote:

I guess I’m still not sure what do do with what comes back.

I’m getting back

order={"status"=>"open", "saledate"=>"2014 May 19 20:02",
"invoicedate"=>"", "shippeddate"=>"", "emaileddate"=>"",
"shippingcost"=>"", "container_id"=>"a9b8faa6-5ea9-11e3-8601-003048686938",
"source_id"=>"9a4a201e-0a7b-11e2-a873-003048686938", "note"=>"",
"items_attributes"=>{"0"=>{"id"=>"3b22c47a-dfcb-11e3-925b-003048686938",
"quantity"=>"1", "sku_id"=>"pgrpt", "name"=>"PennyGems Relative P/T Sheet",
"price"=>"9.75", "shipping"=>"", "discount"=>"", "salestax"=>""}}}

and . . . hey, wait . . . there’s no ID for the order itself. Surely I
don’t have to explicity add the primary key? Oh, no, because even if I do,
it’s still not included in the form. f.input(:id) does not create an
tag.

Sequel doesn't not automatically added a hidden field for the primary key.
The assumption is that's part of the URL you would use in the form tag's
action attribute.

f.input(:id, :type=>:hidden)

should return a hidden input field with the primary key.

Well, then, no wonder I got this error:

Sequel::Error at /orders/savechanges
no matching associated object with given primary key (association:
items, pk: ["3b22c47a-dfcb-11e3-925b-003048686938”]

Because the item with that PK belongs to Order 29823, and I don’t know how
to update that order with these data…

If you are modifying an existing order, you should be doing something like:

order = Order.with_pk!(params[:id]) # assuming the id parameter comes
from the URL.
order.update(params[:order])

You shouldn't be using Order.new unless you are receiving a new order.

Other than that, this kinda worked….

            order = Order.new(request.params["order"]) 
            order.save 

Although I suspect I really need to do a .create_or_update

It's generally a bad idea to try to have the same code handle creating new
orders and updating existing orders. Though if you want to do that, Sequel
has an update_or_create plugin that makes it easier.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 20 May 2014 15:24:13 -0700
Google Group Sender: [email protected]

On May 20, 2014, at 8:23 , Jeremy Evans [email protected] wrote:

Sequel doesn't not automatically added a hidden field for the primary key. The assumption is that's part of the URL you would use in the form tag's action attribute.

Er, let me double-check that. It "doesn’t not” automatically add? Sequel doesn’t? I think what you’re trying to tell me is that Forme’s Sequel plug-in doesn’t include a hidden PK input for the root object of the form, but I’d like to confirm that...

order = Order.with_pk!(params[:id]) # assuming the id parameter comes from the URL.

.with_pk!? I don’t recall that one. You must have snuck it in on me . . . hmm.

You shouldn't be using Order.new unless you are receiving a new order.

yea, I told you I was pretty sure I wasn’t doing it right. :)

It's generally a bad idea to try to have the same code handle creating new orders and updating existing orders. Though if you want to do that, Sequel has an update_or_create plugin that makes it easier.

No, because I’m almost positive it’s not possible to invoke this form until after an Order has been created. THere’s a “make new invoice for buyer X” which instantiates a new one and then let’s you edit it, and there’s a “duplicate existing” option. So it will always be an update.

[jeremyevans]

Google Group Date: Tue, 20 May 2014 17:59:36 -0700
Google Group Sender: [email protected]

On Tuesday, May 20, 2014 3:24:13 PM UTC-7, Dave Howell wrote:

On May 20, 2014, at 8:23 , Jeremy Evans wrote:

Sequel doesn't not automatically added a hidden field for the primary
key. The assumption is that's part of the URL you would use in the form
tag's action attribute.

Er, let me double-check that. It "doesn’t not” automatically add? Sequel
doesn’t? I think what you’re trying to tell me is that Forme’s Sequel
plug-in doesn’t include a hidden PK input for the root object of the form,
but I’d like to confirm that...

Correct.

order = Order.with_pk!(params[:id]) # assuming the id parameter comes
from the URL.

.with_pk!? I don’t recall that one. You must have snuck it in on me . . .
hmm.

That's been part of Sequel for over a year. :)

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 20 May 2014 18:18:04 -0700
Google Group Sender: [email protected]

On May 20, 2014, at 17:59 , Jeremy Evans [email protected] wrote:

.with_pk!? I don’t recall that one. You must have snuck it in on me . . . hmm.

That's been part of Sequel for over a year. :)

Only a year? I’ve been using Sequel longer than that. So you DID sneak it in.

Now the question is, what’s it for?

I have, as of just a moment or two ago, actually altered a record! Wooooohooooo!

Here’s the code I used to handle the returned data:

orderdata = request.params["order"]
orderdata.delete_empty!
order_id = orderdata.delete("id").to_i
order = Order[order_id] # assuming the id parameter comes from the URL.
order.update(orderdata)
orderdata["items_attributes"].each do |k,v|
itemdata = v
itemdata.delete_empty!
item_id = itemdata.delete("id")
Item[item_id].update(itemdata)
end

Hash.delete_empty! does what you’d think: any key where theHash[key].empty? is true, are deleted from the hash.

Yes, I know, that means I can never put a blank string in my database. I don’t have a problem with that.

What I’ll probably do shortly is rewrite this more like my existing one, where the catching code is generalized. If exec(request.param.key) = Class and request.param[“id”].exist? then (attempt to instantiate that record, handle as above). If key.match(/_attributes$/), call self recursively . . . .

[jeremyevans]

Google Group Date: Tue, 20 May 2014 18:25:48 -0700
Google Group Sender: [email protected]

On Tuesday, May 20, 2014 6:18:04 PM UTC-7, Dave Howell wrote:

What I’ll probably do shortly is rewrite this more like my existing one,
where the catching code is generalized. If exec(request.param.key) = Class
and request.param[“id”].exist? then (attempt to instantiate that record,
handle as above). If key.match(/_attributes$/), call self recursively . . .
.

I would be very careful to make sure that you aren't opening yourself up to
security issues when doing that. It sounds like your are describing a
design which would allow a user modify any Sequel::Model object in your
database. You probably want to use a whitelisting approach so that you
limit what changes are allowed.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 20 May 2014 18:33:37 -0700
Google Group Sender: [email protected]

On May 20, 2014, at 18:25 , Jeremy Evans [email protected] wrote:

On Tuesday, May 20, 2014 6:18:04 PM UTC-7, Dave Howell wrote:
What I’ll probably do shortly is rewrite this more like my existing one, where the catching code is generalized. If exec(request.param.key) = Class and request.param[“id”].exist? then (attempt to instantiate that record, handle as above). If key.match(/_attributes$/), call self recursively . . . .

I would be very careful to make sure that you aren't opening yourself up to security issues when doing that. It sounds like your are describing a design which would allow a user modify any Sequel::Model object in your database. You probably want to use a whitelisting approach so that you limit what changes are allowed.

Ah. Good point. Now, this particular database is a one-user database; it’s just me. But point taken.

My current system generates a one-time pad for each form to track the objects it sends out in order to figure out what to update when it comes back. Off the top of my head, that would seem to offer a means to prevent arbitrary code injection.

Most of my primary keys are already UUIDs, which would make it extraordinarly difficult to modify records other than those used to build the form even if I didn’t replace actual IDs with short-term references to them. Yes? No?

[jeremyevans]

Google Group Date: Tue, 20 May 2014 18:41:23 -0700
Google Group Sender: [email protected]

On Tuesday, May 20, 2014 6:33:37 PM UTC-7, Dave Howell wrote:

On May 20, 2014, at 18:25 , Jeremy Evans wrote:

On Tuesday, May 20, 2014 6:18:04 PM UTC-7, Dave Howell wrote:
What I’ll probably do shortly is rewrite this more like my existing one,
where the catching code is generalized. If exec(request.param.key) = Class
and request.param[“id”].exist? then (attempt to instantiate that record,
handle as above). If key.match(/_attributes$/), call self recursively . . .
.

I would be very careful to make sure that you aren't opening yourself up
to security issues when doing that. It sounds like your are describing a
design which would allow a user modify any Sequel::Model object in your
database. You probably want to use a whitelisting approach so that you
limit what changes are allowed.

Ah. Good point. Now, this particular database is a one-user database; it’s
just me. But point taken.

My current system generates a one-time pad for each form to track the
objects it sends out in order to figure out what to update when it comes
back. Off the top of my head, that would seem to offer a means to prevent
arbitrary code injection.

I'd say it's unlikely that simply using a one-time pad would make things
secure, but you would need to describe in more detail what you are doing.
Note that the issue is not necessarily arbitrary code injection, just the
ability to access/modify objects that you didn't intend to allow.

Most of my primary keys are already UUIDs, which would make it
extraordinarly difficult to modify records other than those used to build
the form even if I didn’t replace actual IDs with short-term references to
them. Yes? No?

UUIDs don't grant any additional security, they just make guessing the
primary keys more difficult. If you are leaking that information anywhere
else, then the attacker doesn't need to guess. It's probably a better idea
to make sure your system is secure even if the attacker knows the primary
key values.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 20 May 2014 19:09:26 -0700
Google Group Sender: [email protected]

On May 20, 2014, at 18:41 , Jeremy Evans [email protected] wrote:

UUIDs don't grant any additional security, they just make guessing the primary keys more difficult. If you are leaking that information anywhere else, then the attacker doesn't need to guess. It's probably a better idea to make sure your system is secure even if the attacker knows the primary key values.

Well, yes, but since the default is sequential integers, that’s a huge difference already.

What I’m imagining with the pad is that the end effect is that a record can only be altered if there’s a valid reference to it in the pad table, and the pads expire after a limited amount of time. So when Forme (or the surrounding code) builds a form, it ‘unlocks’ those specific records by creating the referencing codes, and when the form comes back, it uses those codes to find the actual PKs. Once the pad reference expires, shoving forms at the server won’t grant access to modify data. If you’re using UUIDs for pads, trying to guess them randomly is also pointless.

A further obvious step is that the code that does the submitting also wipes the pad from the table. Thus, once an HTML form is built, it can only be submitted once. So if you’re trying to do unauthorized modifications, you’ve got to intercept the pad and submit before the legitimate user does.

[jeremyevans]

Google Group Date: Tue, 20 May 2014 19:24:14 -0700
Google Group Sender: [email protected]

On Tuesday, May 20, 2014 7:09:26 PM UTC-7, Dave Howell wrote:

What I’m imagining with the pad is that the end effect is that a record
can only be altered if there’s a valid reference to it in the pad table,
and the pads expire after a limited amount of time. So when Forme (or the
surrounding code) builds a form, it ‘unlocks’ those specific records by
creating the referencing codes, and when the form comes back, it uses those
codes to find the actual PKs. Once the pad reference expires, shoving forms
at the server won’t grant access to modify data. If you’re using UUIDs for
pads, trying to guess them randomly is also pointless.

This means that requesting a form display requires inserting rows into your
database. This is problematic as depending on the number of inserted rows
(1 row per object, 1 row per column per object), you can open yourself up
to a denial of service attack by an attacker repeatedly requesting the page
containing the form over and over. If you try to get smart and delete
previous inserted rows if the form display is requested again, then you
break the stateless nature of the web.

In general you want your idempotent web requests (i.e. anything requested
via GET) is idempotent from a database perspective as well (SELECT only).

A further obvious step is that the code that does the submitting also
wipes the pad from the table. Thus, once an HTML form is built, it can only
be submitted once. So if you’re trying to do unauthorized modifications,
you’ve got to intercept the pad and submit before the legitimate user does.

You are assuming the wrong thing. The attacker is not a third-party
running a man in the middle attack on the user (or you've got bigger
issues). The attacker is the user. If you want to design a secure
application, you need to assume every user of the app is a potential
attacker.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 20 May 2014 21:41:12 -0700
Google Group Sender: [email protected]

On May 20, 2014, at 19:24 , Jeremy Evans [email protected] wrote:

On Tuesday, May 20, 2014 7:09:26 PM UTC-7, Dave Howell wrote:

You are assuming the wrong thing. The attacker is not a third-party running a man in the middle attack on the user (or you've got bigger issues). The attacker is the user. If you want to design a secure application, you need to assume every user of the app is a potential attacker.

Okay, I'll assume the user is the source of the attack.

This started with my comment about generalizing the form data processing code. If the one-time-use pad is granting write-once access to Model objects that I’ve sent to the user to modify in the first place, in what way is the generalized code any less secure than

order.update(orderdata)

where I’m taking whatever came back and letting Sequel match up the parts? The only avenue for attack I can think of even for my original code using the actual primary keys is if somebody keeps track of the pk for each record or associated record they touch, so that they can submit modifications to it later, and in most cases, they could just go back to that page on the web site and change it with the same page that they used to find out what the UUID was in the first place.

I should also confess that I have almost never been so fortunate as to have to worry about a site’s performance. I’ve always focused on making it work as well as I could: if something became so popular that the code or server couldn’t handle it, well, it’s a bridge I’ve never had to cross. However, I don’t think it’d be necessary to keep the one-time key -> primary key lookup table in the database. It could be an in-memory SQLite table, or a session variable table.

As originally described, it would admittedly be true that you could create imaginary associations and submit them with the form. I don’t think it would be more than a line or two of code to include, when going back around to update an associated record, the id of the parent object. If the associated object doesn’t have a foreign key back to the parent, then it’s bogus and you toss it.

I’m still hoping to write a tutorial for Forme at some point, and it will certainly be sadly incomplete if it doesn’t include something about how to deal with what you get back. I’ll keep these last few emails archived so I can refer back to them when I get to that point, and, of course, we can refine things further.

[jeremyevans]

Google Group Date: Tue, 20 May 2014 22:06:47 -0700
Google Group Sender: [email protected]

On Tuesday, May 20, 2014 9:41:12 PM UTC-7, Dave Howell wrote:

On May 20, 2014, at 19:24 , Jeremy Evans wrote:

On Tuesday, May 20, 2014 7:09:26 PM UTC-7, Dave Howell wrote:

You are assuming the wrong thing. The attacker is not a third-party
running a man in the middle attack on the user (or you've got bigger
issues). The attacker is the user. If you want to design a secure
application, you need to assume every user of the app is a potential
attacker.

Okay, I'll assume the user is the source of the attack.

This started with my comment about generalizing the form data processing
code. If the one-time-use pad is granting write-once access to Model
objects that I’ve sent to the user to modify in the first place, in what
way is the generalized code any less secure than

order.update(orderdata)

where I’m taking whatever came back and letting Sequel match up the parts?
The only avenue for attack I can think of even for my original code using
the actual primary keys is if somebody keeps track of the pk for each
record or associated record they touch, so that they can submit
modifications to it later, and in most cases, they could just go back to
that page on the web site and change it with the same page that they used
to find out what the UUID was in the first place.

Using a one-time pad as you describe is no less secure, but it isn't any
more secure, either. Since it doesn't add to security, just complexity,
it's a net negative.

As originally described, it would admittedly be true that you could create
imaginary associations and submit them with the form. I don’t think it
would be more than a line or two of code to include, when going back around
to update an associated record, the id of the parent object. If the
associated object doesn’t have a foreign key back to the parent, then it’s
bogus and you toss it.

That's basically how Sequel's nested_attributes plugin handles the issue.

Note that In general the best way to handle form submissions with Sequel is
to do:

order.update_fields(order_data, [:allowed_field1, :allowed_field2, ...])

In this way, the user can only change fields you specifically allow for
that action, as opposed to any of the non-primary fields (or any of the
allowed columns if you are using set_allowed_columns).

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Fri, 30 May 2014 14:13:50 -0700
Google Group Sender: [email protected]

I was on vacation. :) I’m back now, and ready to pester you some more. Heh heh.

Let us say for the sake of argument (or, rather, for the sake of non-argument), that it not advisable to have generalized form-processing code on a production server, or otherwise deployed to mere users. I would suggest that it would nevertheless be useful to have such code for tutorial purposes. To create a method that can take (almost) anything that comes back from Forme, particularly what comes back from the Sequel plug-in, means it would be a block of code that would illuminate how to process a wide variety of Forme forms.

I also see some benefit, if I’m doing development work, to being able to tinker and experiment with my Formes and not have to worry about rewriting the receiving code. I would like to come up with such a method, both for my own one-man inventory system, and for the tutorial I’m hoping to write at some point. The tutorial would definitely have clear warnings about the inadvisability of using it in the ‘real world.’

So, I sat down to whip something up, and hit a roadblock on the first line of code. {sigh} Actually, I think the middle part is the easy part; it’s the outer shell that’s pretty instantly extra-meta-ish.

It goes something like this:

*La la la. Oh, hey, I’ve been invoked. That means I’m supposed to insert or update the database. I will need to make some Sequel::Models, and then call .update or .create on them (as needed)....*

It just occurred to me that I’d planned to work with the request.params hash, but that might be fairly idiosyncratic to Ramaze or Rack or some such. Do you think it would be better to work directly with the POST string so as to be more universal, or is it a reasonable assumption to expect a hash? Also, I’d been thinking of making this a destination, but now that I think about it, I’m leaning more toward having it be an internal call.

Option A is to have all (or most) of the forms have the same action, and include an <input type=hidden name=returnto value=“/my/webpage/display?id=#{obj.id}”>. That’s where I was going, but now I’m thinking it’d be better to do Option B: each form goes to its own little destination, which mostly looks like
def destination
processForme(request.params)
redirect("/my/webpage/display?id=#{obj.id}”)
end

The roadblock mentioned earlier is because Option A can’t know much more than “request.params”. If I’m doing Option B, I could make the call more like
def destination
processForme(:objectclass, request.params)
redirect("/my/webpage/display?id=#{obj.id}”)
end
Let’s call that one Option C. The code for processForme is certainly simpler for Option C, but C doesn’t have the universality I’d like to have, so for the ‘tinkering with your Forme during development’ use, I’d like to try to do Option B. So, back to the inner dialog of processForme . . . .

*…But first I have to figure out _which_ Sequel::Models I’m going to need. So here’s this hash….*

The request.params hash that I’ve been seeing on my system when I use a Forme form for my order wtih its associated items is something like
{:sessionKey => “greatbiglongthingthatRamazeUses", “order" => {:id => ‘orderID’, :key => “appropriate Value”, . . .}}

*…so here’s this hash. Let’s see which of the keys, if any, are Sequel models. Does eval(:sessionKey.to_s.capitalize) == Sequel::Model? Nope. Does eval(“order”.to_s.capitalize) == Sequel::Model? Yes it does! And what is the primary key(s) for that model? Ah, “:id”. Do I have an :id in the hash value for the “order” key? I do. If I instantiate Order[formHash[“order”][:id]], is the result not nil? It is. I’m doing an update!*

I think I would eventually be able to figure out how to create a working expression for "eval(:sessionKey.to_s.capitalize) == Sequel::Model”, but I’m also pretty sure that’s not it. I’m also willing to bet that you know how to write that just off the top of your head. Yes, it’s actually more like request.params.keys.select {|key| key.to_s.capitalize.defined? and eval(key.to_s.capitalize) == Sequel::Model}, but I figure you can see what I’m getting at here. 

Hints?

[jeremyevans]

Google Group Date: Fri, 30 May 2014 22:04:26 -0700
Google Group Sender: [email protected]

On Friday, May 30, 2014 2:13:54 PM UTC-7, Dave Howell wrote:

I was on vacation. :) I’m back now, and ready to pester you some more. Heh
heh.

Let us say for the sake of argument (or, rather, for the sake of
non-argument), that it not advisable to have generalized form-processing
code on a production server, or otherwise deployed to mere users. I would
suggest that it would nevertheless be useful to have such code for tutorial
purposes. To create a method that can take (almost) anything that comes
back from Forme, particularly what comes back from the Sequel plug-in,
means it would be a block of code that would illuminate how to process a
wide variety of Forme forms.

I don't think this is a good idea, as it teaches people the wrong thing.
You can't generalize the form processing, the form processing code should
be specific to the form you create or you are going to have security
issues. The only way you could make generalized form processing secure is
if you start cryptographically signing the forms, and I don't want to go
there. Ignoring security issues when teaching people how to use an API
results in insecure usage of that API by the people learning.

However, I do see some benefits to showing people examples of how to write
form processing code. Such as:

1. New model object creation:

o = Model.new
o.update_fields(params[:model], [:field1, :field2, ...])

2. Existing model object update:

o = Model.with_pk!(params[:id])
o.update_fields(params[:model], [:field1, :field2, ...])

It just occurred to me that I’d planned to work with the request.params

hash, but that might be fairly idiosyncratic to Ramaze or Rack or some
such. Do you think it would be better to work directly with the POST string
so as to be more universal, or is it a reasonable assumption to expect a
hash? Also, I’d been thinking of making this a destination, but now that I
think about it, I’m leaning more toward having it be an internal call.

100% you should be working on a hash, not on the POST data or GET query
string.

Option A is to have all (or most) of the forms have the same action, and
include an <input type=hidden name=returnto value=“/my/webpage/display?id=#{
obj.id}”>. That’s where I was going, but now I’m thinking it’d be better
to do Option B: each form goes to its own little destination, which mostly
looks like
def destination
processForme(request.params)
redirect("/my/webpage/display?id=#{obj.id}”)
end

Option B is much better, since the redirect control is on the server and
not on the client. But again, a generic method like processForme isn't
going to work.

The roadblock mentioned earlier is because Option A can’t know much more
than “request.params”. If I’m doing Option B, I could make the call more
like
def destination
processForme(:objectclass, request.params)
redirect("/my/webpage/display?id=#{obj.id}”)
end
Let’s call that one Option C. The code for processForme is certainly
simpler for Option C, but C doesn’t have the universality I’d like to have,
so for the ‘tinkering with your Forme during development’ use, I’d like to
try to do Option B. So, back to the inner dialog of processForme . . . .

    *…But first I have to figure out _which_ Sequel::Models I’m going 

to need. So here’s this hash….*

The request.params hash that I’ve been seeing on my system when I use a
Forme form for my order wtih its associated items is something like
{:sessionKey => “greatbiglongthingthatRamazeUses", “order" => {:id
=> ‘orderID’, :key => “appropriate Value”, . . .}}

    *…so here’s this hash. Let’s see which of the keys, if any, are 

Sequel models. Does eval(:sessionKey.to_s.capitalize) == Sequel::Model?
Nope. Does eval(“order”.to_s.capitalize) == Sequel::Model? Yes it does! And
what is the primary key(s) for that model? Ah, “:id”. Do I have an :id in
the hash value for the “order” key? I do. If I instantiate
Order[formHash[“order”][:id]], is the result not nil? It is. I’m doing an
update!*

    I think I would eventually be able to figure out how to create a 

working expression for "eval(:sessionKey.to_s.capitalize) ==
Sequel::Model”, but I’m also pretty sure that’s not it. I’m also willing to
bet that you know how to write that just off the top of your head. Yes,
it’s actually more like request.params.keys.select {|key|
key.to_s.capitalize.defined? and eval(key.to_s.capitalize) ==
Sequel::Model}, but I figure you can see what I’m getting at here.

Hints?

The only hint I have is: Don't try to write a generic method. :)

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Sat, 31 May 2014 01:21:44 -0700
Google Group Sender: [email protected]

As an addenda to the previous message, I have a single form-processing routine for my old form code. Some of the really simple forms have their own methods, but any form that had associated tables was sent to “processForm(formHash)”. Grep tells me I have five forms that use it, all with different primary objects.

[jeremyevans]

Google Group Date: Sat, 31 May 2014 08:14:01 -0700
Google Group Sender: [email protected]

On Saturday, May 31, 2014 1:21:48 AM UTC-7, Dave Howell wrote:

As an addenda to the previous message, I have a single form-processing
routine for my old form code. Some of the really simple forms have their
own methods, but any form that had associated tables was sent to
“processForm(formHash)”. Grep tells me I have five forms that use it, all
with different primary objects.

How can you be sure that the the processing is secure, in the sense that a
malicious user is not able to set fields that do not appear on the related
form?

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Sat, 31 May 2014 15:45:05 -0700
Google Group Sender: [email protected]

On May 31, 2014, at 8:14 , Jeremy Evans [email protected] wrote:

How can you be sure that the the processing is secure, in the sense that a malicious user is not able to set fields that do not appear on the related form?

Because there’s only one user? And since I have (and often use) direct-to-database access, I have no reason to play games with the form.

As I write this, an idea has occurred to me. In order to provide a barrier to exactly that issue, what about if I patched my Forme so that every time .input was called (or maybe whenever it builds an tag), it also registered the input’s name in some kind of session variable, sort of a form manifest. When the Forme-return-processing code ran, it would only process keys on the list. Any extraneous fields inserted by a malicious user would be ignored.

In order to maintain a quality user experience, it might be better to be able to retain multiple form manifests, for people who might open multiple windows onto the site, and, as previously discussed, you could clear a manifest either when the corresponding form was processed, or after a certain amount of time.

It seems to me that there would be a fair amount of merit in having a single well-tested form-return routine rather than writing nearly identical code over and over for multiple forms.

I’m looking over the docs, and that’s leading me to the next question . . . .

[jeremyevans]

Google Group Date: Sat, 31 May 2014 22:58:39 -0700
Google Group Sender: [email protected]

On Saturday, May 31, 2014 3:45:09 PM UTC-7, Dave Howell wrote:

On May 31, 2014, at 8:14 , Jeremy Evans wrote:

How can you be sure that the the processing is secure, in the sense that
a malicious user is not able to set fields that do not appear on the
related form?

Because there’s only one user? And since I have (and often use)
direct-to-database access, I have no reason to play games with the form.

That's a fairly uncommon use case though. The vast majority of developers
using Forme would be using it to build forms that users of their
application (i.e. potential attackers) would be able to use.

As I write this, an idea has occurred to me. In order to provide a barrier
to exactly that issue, what about if I patched my Forme so that every time
.input was called (or maybe whenever it builds an tag), it also
registered the input’s name in some kind of session variable, sort of a
form manifest. When the Forme-return-processing code ran, it would only
process keys on the list. Any extraneous fields inserted by a malicious
user would be ignored.

This would only work in very narrow circumstances. If you have multiple
web application processes, it wouldn't work. If you restarted the
application between when the form was displayed and when it was submitted,
it wouldn't work. As http is a stateless protocol, submission of the form
can come from any previous display of the form.

In order to maintain a quality user experience, it might be better to be
able to retain multiple form manifests, for people who might open multiple
windows onto the site, and, as previously discussed, you could clear a
manifest either when the corresponding form was processed, or after a
certain amount of time.

If you want to support multiple manifests, then you have to store them
server-side somehow, and your form processing code needs to specify which
manifest to use.

It seems to me that there would be a fair amount of merit in having a
single well-tested form-return routine rather than writing nearly identical
code over and over for multiple forms.

I mentioned that if you wanted to do this, the only way to do so securely
is to store the manifest in a hidden form input, and cryptographically sign
it to prevent tampering.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Sun, 1 Jun 2014 01:39:32 -0700
Google Group Sender: [email protected]

On May 31, 2014, at 22:58 , Jeremy Evans [email protected] wrote:

This would only work in very narrow circumstances.

If you restarted the application between when the form was displayed and when it was submitted, it wouldn't work.

I don’t have a problem with that. Even my home web server doesn’t get restarted more than once a month or so.

If you have multiple web application processes, it wouldn't work.

If I have multiple web application processes and I’m load-balancing on a per-request basis instead of per-user basis, it wouldn’t work. But that’s a terrible idea, because it means that NO session variables could be retained. They’d all have to be explicit cookies, or being written to/from the database.

I don’t see your two exceptions here as defining “narrow circumstances.” When I look at my browser’s cookie jar, I rarely see more than two or three cookies per site, and usually one of them’s some excessively long session ID, which is pretty obviously used to as a key for server-side session data. The manifest would be valid no more nor less often than my Amazon shopping cart or American Express banking access.

I mentioned that if you wanted to do this, the only way to do so securely is to store the manifest in a hidden form input, and cryptographically sign it to prevent tampering.

How is that more secure than an inaccessible server-side manifest? Oh, I see, it’s not more secure, it’s that you think it won’t work unless the manifest travels with the form itself.

We might just have to agree to disagree on this, because I think what you call “narrow circumstances” I would call “98% of everybody who would be trying to learn Forme,” or at least “98% of the people I expect would need a good tutorial for Forme.” Back when I had to store all of my session data in the database (SQLServer 6.5) and my web server was a Pentium II, I could still easily handle 20+ pages a second. If I’m lucky, then before the end of the year I’ll get to start working on a project that could, if everything goes well, eventually have enough traffic that I might actually have to care about performance. Until then, having one form-return routine will save me time, reduce code redundancy, reduce the bug count, simplify testing, and enhance my security.

In the last fifteen years, the biggest/busiest site I’ve had to code for was for a pharmaceutical manufacturer. There were no more than 20 authorized users, and both the database and the web server were running in the background on a Mac Mini in the office. The IT budget was so small that getting the most functionality from the least amount of programming time and with the fewest possible bugs was paramount. If I’d had Forme back then, I’d have leaped right to it, and if there was also a generic form-return-processing routine, even if it was completely without any kind of security, I’d have snatched that up in a heartbeat as well. The value of being able to use tested, debugged, and documented code to handle the form returns would utterly, totally dwarf the risk that one of the lab technicians might try to hack the database by building bogus forms. If they’d had anybody even remotely savvy enough to try that, they wouldn’t have needed me in the first place, and if I were such a user, I would have just worked on getting root access to the server. Even w/o physical access (which makes it trivial), it still wouldn’t have been all that hard.

Should the day come that my code is still being used on a site that is so busy that a server-side manifest becomes problematic, then that would be the time to replace the manifest storage and retrieval code with one that used encryption and stored it in the form itself, an upgrade that will be much easier if all the forms on the site are driven by the same code.

For me at least, Forme without Sequel is pointless. It’s easier and faster to just write a form in HTML than to figure out how to use Forme to build it. Database-driven on-the-fly form creation starts to become useful. On-the-fly forms that handle associations? That’s the big win; that’s what can save me significant time and many bugs, and that’s why I wrote my own library to do just that. But a library that turns database data into forms, but doesn’t turn forms back into database data, is incomplete. One cannot NOT have the other half; it’s completely pointless to make forms if you don’t do anything with the return. Do you really think every programmer using Forme should build the other half from scratch, and reinvent the wheel over and over again? I see far more virtue in including the return code as part of the package, code that will have been written and tested by multiple people, that will eventually represent hundreds or thousands of hours of field use and experience funneled back to make code far more robust and flexible than what any one programmer might come up with on their own.

To do that, though, it cannot be anything but universal, because every application has different objects. My very first Forme return code hard-coded for the objects to process. “The primary object is an Order. Update the database with the form hash for that. If there are any associated Items, do those too.” The utility and functionality would have been absolutely identical if I could have handled with with a call to Forme instead, along the lines of “Dear Forme, here’s the hash of a form. Please process an Order object, and any associated Items.” In other words, calling it with an explicit, and hard-coded, manifest.

If I wanted to bring up a form that would allow me to modify the data for Customer, and every Order they’ve had, including every Item in each order, I could build the manifest as “{:customer => {:order => :item}}”, and it would also be every bit as good as the code that I came up with. Well, every bit as good until/unless it includes any error checking. At that point , it’s just flat-out better.

Once that exists, then offering an optional plug-in to build the manifest on the fly seems obvious, and might well get written by somebody else. If that plug-in then gets the additional code necessary for in-form encrypted transport of the manifest, it becomes enormously superior to anything I would ever have the time to write.

{pfoomph!}

Oh!

Okay, this message is bloated enough. Interesting new idea on next rock . . . .

[jeremyevans]

Google Group Date: Sun, 1 Jun 2014 16:41:38 -0700
Google Group Sender: [email protected]

On Sunday, June 1, 2014 1:39:36 AM UTC-7, Dave Howell wrote:

On May 31, 2014, at 22:58 , Jeremy Evans wrote:

This would only work in very narrow circumstances.

If you restarted the application between when the form was displayed
and when it was submitted, it wouldn't work.

I don’t have a problem with that. Even my home web server doesn’t get
restarted more than once a month or so.

If you have multiple web application processes, it wouldn't work.

If I have multiple web application processes and I’m load-balancing on a
per-request basis instead of per-user basis, it wouldn’t work. But that’s a
terrible idea, because it means that NO session variables could be
retained. They’d all have to be explicit cookies, or being written to/from
the database.

Virtually all ruby web applications are going to be load balanced per
request, not per user. I don't think you can find a well known ruby
developer that would publicly advocate for storing session information in
process memory. In general with ruby web applications (the default in
Rails and Sinatra at least), session variables will be stored in cookies by
default, at least cryptographically signed, and possibly encrypted. The
few applications that are not storing session information in a cookie are
probably storing it in some shared datastore (SQL database, redis, etc.).

I don’t see your two exceptions here as defining “narrow circumstances.”
When I look at my browser’s cookie jar, I rarely see more than two or three
cookies per site, and usually one of them’s some excessively long session
ID, which is pretty obviously used to as a key for server-side session
data. The manifest would be valid no more nor less often than my Amazon
shopping cart or American Express banking access.

I mentioned that if you wanted to do this, the only way to do so
securely is to store the manifest in a hidden form input, and
cryptographically sign it to prevent tampering.

How is that more secure than an inaccessible server-side manifest? Oh, I
see, it’s not more secure, it’s that you think it won’t work unless the
manifest travels with the form itself.

Correct. Because of the stateless nature of http. Any time you are are
storing the information server side, you have one of two problems:

1. If you clear the session store of stale objects, you break the stateless
   nature of the http.

2. If you don't clear the session store of stale objects, you open yourself
   up to denial of service.

We might just have to agree to disagree on this, because I think what you
call “narrow circumstances” I would call “98% of everybody who would be
trying to learn Forme,” or at least “98% of the people I expect would need
a good tutorial for Forme.” Back when I had to store all of my session data
in the database (SQLServer 6.5) and my web server was a Pentium II, I could
still easily handle 20+ pages a second. If I’m lucky, then before the end
of the year I’ll get to start working on a project that could, if
everything goes well, eventually have enough traffic that I might actually
have to care about performance. Until then, having one form-return routine
will save me time, reduce code redundancy, reduce the bug count, simplify
testing, and enhance my security.

In the last fifteen years, the biggest/busiest site I’ve had to code for
was for a pharmaceutical manufacturer. There were no more than 20
authorized users, and both the database and the web server were running in
the background on a Mac Mini in the office. The IT budget was so small that
getting the most functionality from the least amount of programming time
and with the fewest possible bugs was paramount. If I’d had Forme back
then, I’d have leaped right to it, and if there was also a generic
form-return-processing routine, even if it was completely without any kind
of security, I’d have snatched that up in a heartbeat as well. The value of
being able to use tested, debugged, and documented code to handle the form
returns would utterly, totally dwarf the risk that one of the lab
technicians might try to hack the database by building bogus forms. If
they’d had anybody even remotely savvy enough to try that, they wouldn’t
have needed me in the first place, and if I were such a user, I would have
just worked on getting root access to the server. Even w/o physical access
(which makes it trivial), it still wouldn’t have been all that hard.

We are going to have to agree to disagree. I see security as of the utmost
importance, and think the tradeoffs you are describing in the name of
simplicity do more harm than good.

Should the day come that my code is still being used on a site that is so
busy that a server-side manifest becomes problematic, then that would be
the time to replace the manifest storage and retrieval code with one that
used encryption and stored it in the form itself, an upgrade that will be
much easier if all the forms on the site are driven by the same code.

For me at least, Forme without Sequel is pointless. It’s easier and faster
to just write a form in HTML than to figure out how to use Forme to build
it. Database-driven on-the-fly form creation starts to become useful.
On-the-fly forms that handle associations? That’s the big win; that’s
what can save me significant time and many bugs, and that’s why I wrote my
own library to do just that. But a library that turns database data into
forms, but doesn’t turn forms back into database data, is incomplete. One
cannot NOT have the other half; it’s completely pointless to make forms if
you don’t do anything with the return. Do you really think every programmer
using Forme should build the other half from scratch, and reinvent the
wheel over and over again? I see far more virtue in including the return
code as part of the package, code that will have been written and tested by
multiple people, that will eventually represent hundreds or thousands of
hours of field use and experience funneled back to make code far more
robust and flexible than what any one programmer might come up with on
their own.

Find me another ruby form library that includes support for handling the
request data in a way similar to you are describing. I don't think there
are any, and that's because most developers consider the issues completely
separate. Form libraries are for building html forms, they are not for
handling request data.

In a stateful application/protocol, it would make sense to have a form
class that also handled form submission, since you could make the request
handling a different method on the same Form instance. However, http is a
stateless protocol, when the request comes in, you don't have access to the
same Form instance, so it does not make sense to do so.

Attempting to write a generic form processing method without access to the
same instance that created the form (or the ability to get the same
metadata stored in the Form object that created the html form) is
problematic for reasons I already stated. Storing the manifest server-side
has issues as I already stated. Storing the manifest in the form itself is
only safe if you encrypt/sign it, as I already stated. The only real
solution is to store such a manifest in the form itself and encrypt/sign
it, as I already stated.

To do that, though, it cannot be anything but universal, because every
application has different objects. My very first Forme return code
hard-coded for the objects to process. “The primary object is an Order.
Update the database with the form hash for that. If there are any
associated Items, do those too.” The utility and functionality would have
been absolutely identical if I could have handled with with a call to Forme
instead, along the lines of “Dear Forme, here’s the hash of a form. Please
process an Order object, and any associated Items.” In other words, calling
it with an explicit, and hard-coded, manifest.

If I wanted to bring up a form that would allow me to modify the data for
Customer, and every Order they’ve had, including every Item in each order,
I could build the manifest as “{:customer => {:order => :item}}”, and it
would also be every bit as good as the code that I came up with. Well,
every bit as good until/unless it includes any error checking. At that
point , it’s just flat-out better.

Once that exists, then offering an optional plug-in to build the manifest
on the fly seems obvious, and might well get written by somebody else. If
that plug-in then gets the additional code necessary for in-form encrypted
transport of the manifest, it becomes enormously superior to anything I
would ever have the time to write.

There has been talk in other ruby form libraries of dealing with
signed/encrypted manifests and handling form processing automatically. I'm
not aware of any successful attempts to do so. I'm not saying it's
impossible, but it's not going to be an easy task. Since writing form
processing code is generally very easy to do manually, writing a lot of
very difficult and security-risky code to make form processing code
slightly easier does not seem like a good use of time to me, so it is not
something I plan to work on.

I'll finish this by stating that even if you could come up with such a
manifest feature, it could not possibly handle all cases. Consider any
html form where fields can be added/removed via javascript, as a simple
example. Yes, I've used Forme to build such forms.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Mon, 2 Jun 2014 19:02:02 -0700
Google Group Sender: [email protected]

On Jun 1, 2014, at 16:41 , Jeremy Evans [email protected] wrote:

We are going to have to agree to disagree. I see security as of the utmost importance, and think the tradeoffs you are describing in the name of simplicity do more harm than good.

Check. I’m rather dubious about the use of “utmost.” I cannot believe you would would assign a higher value to a site that was extremely secure but so dysfunctional as to be effectively useless, over one that was functional but not particularly secure, especially if the information in question was not particularly valuable. However, I do not claim that the use case I’ve been arguing for would meet that criteria.

If I amend that to you seeing security as extremely important, it still gives me (I think) a better perspective on your concerns. I hope we can at least agree that (a) there are plenty of existing web sites that should (and hopefully do) consider site security to be extremely important, and that (b) there is at least one site (my own internal one-person order management site) where securing form submission is not as important.

It would thus seem to me that whether or not universal code should be made available publically would mostly be a question of (1) how much less importance form security would be for a very-low-user-count limited-access site (0% < importance <= importance_in_general), and (2) how many such sites there are.

I have found your arguments for the answer for #2 being “too few to justify preparing an automatic manifest-building form return library ready for release” convincing.

Find me another ruby form library that includes support for handling the request data in a way similar to you are describing. I don't think there are any, and that's because most developers consider the issues completely separate. Form libraries are for building html forms, they are not for handling request data.

Oh, now, you’re better than that. I lost count years ago of how many times somebody’s told me “Nobody else does it like that; why should we?” and been dead wrong. I only found three Ruby form libraries, period, and one of them didn’t seem to be particularly good.

After all the time I’ve spent learning from you, I have a better idea of where to look . . . . and I’ve found about fifteen. Two of them state support for nested models: cocoon and nested_form. Both of them are Rails-specific and require jQuery, and I think both could necessitate modifications to the Models used. I don’t know something (Rails? jQuery? I can’t tell) well enough to be able to figure out how the returns are handled.

Do YOU consider them “completely separate?” Since it is practically impossible to process a form if you don’t know anything about the fields you’re getting back, I can’t imagine how anybody could reach such a conclusion. I can imagine somebody not bothering with form-return code because they think “heck, that’s the easy part,” and if you’re not nesting, yea, it is.

As I said earlier, what I saw an obvious opportunity for was code that was smart enough to use the metadata available from the database to make sophisticated nested-model forms without requiring a lot of complicated programming. I did not find anything that appeared to even attempt that besides my own code, Forme, and crushyform.

I'll finish this by stating that even if you could come up with such a manifest feature, it could not possibly handle all cases. Consider any html form where fields can be added/removed via javascript, as a simple example. Yes, I've used Forme to build such forms.

Oh, I know. There will always be exceptional cases that require custom code.

Anyway, I am prepared to abandon my pursuit of form-return code that uses a manifest that was constructed by the form generating code. At least for now. :)

[jeremyevans]

Google Group Date: Mon, 2 Jun 2014 22:07:15 -0700
Google Group Sender: [email protected]

On Monday, June 2, 2014 7:02:07 PM UTC-7, Dave Howell wrote:

On Jun 1, 2014, at 16:41 , Jeremy Evans wrote:

We are going to have to agree to disagree. I see security as of the
utmost importance, and think the tradeoffs you are describing in the name
of simplicity do more harm than good.

Check. I’m rather dubious about the use of “utmost.” I cannot believe you
would would assign a higher value to a site that was extremely secure but
so dysfunctional as to be effectively useless, over one that was
functional but not particularly secure, especially if the information in
question was not particularly valuable. However, I do not claim that the
use case I’ve been arguing for would meet that criteria.

It's better to be secure and have usability issues, than to be insecure and
have no usability issues. That's not an opinion shared by very many
developers, which is one of the reasons so much software we have these days
is insecure.

Find me another ruby form library that includes support for handling the
request data in a way similar to you are describing. I don't think there
are any, and that's because most developers consider the issues completely
separate. Form libraries are for building html forms, they are not for
handling request data.

Oh, now, you’re better than that. I lost count years ago of how many times
somebody’s told me “Nobody else does it like that; why should we?” and been
dead wrong. I only found three Ruby form libraries, period, and one of them
didn’t seem to be particularly good.

After all the time I’ve spent learning from you, I have a better idea of
where to look . . . . and I’ve found about fifteen. Two of them state
support for nested models: cocoon and nested_form. Both of them are
Rails-specific and require jQuery, and I think both could necessitate
modifications to the Models used. I don’t know something (Rails? jQuery? I
can’t tell) well enough to be able to figure out how the returns are
handled.

If you find something that actually handles it, please let me know, as I
would be eager to look at how they implemented it.

Do YOU consider them “completely separate?” Since it is practically
impossible to process a form if you don’t know anything about the fields
you’re getting back, I can’t imagine how anybody could reach such a
conclusion. I can imagine somebody not bothering with form-return code
because they think “heck, that’s the easy part,” and if you’re not nesting,
yea, it is.

Well, semantically, they are related, of course. It's more that I think
they should be treated completely separately.

The idea that there is a 1-1 relationship between form fields and columns
in a database is probably wrong except in simple cases. Consider a user
class, where you are storing a password. You should be storing the
password using some sort of crytographic hash, but you are not going to
display that hash to the user if they are changing their password.

For the simple cases where there is a 1-1 relationship, a manifest idea
would work fine. However, in those cases, writing the form processing code
yourself is trivial. For non-simple cases, you are going to have to write
manual form processing code anyway, as a manifest wouldn't be able to
handle the situation. Because of this, I don't think there is a
significant benefit for manifests, and I see writing the code necessary to
securely handle manifests as a significant cost.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 3 Jun 2014 13:47:31 -0700
Google Group Sender: [email protected]

On Jun 2, 2014, at 22:07 , Jeremy Evans [email protected] wrote:

Well, semantically, they are related, of course. It's more that I think they should be treated completely separately.

The idea that there is a 1-1 relationship between form fields and columns in a database is probably wrong except in simple cases.

I agree. And even when there is a 1:1 relationship, that relationship itself might not be simple.

For the simple cases where there is a 1-1 relationship, a manifest idea would work fine. However, in those cases, writing the form processing code yourself is trivial. For non-simple cases, you are going to have to write manual form processing code anyway, as a manifest wouldn't be able to handle the situation. Because of this, I don't think there is a significant benefit for manifests, and I see writing the code necessary to securely handle manifests as a significant cost.

Uncle! I have abandoned hope of having Forme automatically construct a manifest.

On the other hand, I think there’s still a very strong case for pairing Forme with Emorf, or whatever one might call code to handle the returned form data. I think it’s possible to retain the majority of the benefits I cited, without compromising the security you value, and without having to encrypt a manifest or do anything else along the lines of sneaking fancy stuff into the form.

Note that this is quite independent of the idea of signing the form.

Further, to make it clear that I don’t expect this form-return code to be able to handle every conceivable case, I’m going to call it “nested multi-object form return handling code” rather than “universal” or “generic” code.

The trivial case is Object.update(formHash); so trivial that you wouldn’t bother normalizing it. But does that even work? Assume the hash contains the primary key (I’m a little rushed today or I’d test it to see what happens); will Sequel update the row referenced by the primary key or throw an error?

That’s why my current code extracts the primary key from the hash, then calls
Object[primary_key_value].update(formHash)

This will raise an error if Object[primary_key_value] returns nil, so we ought to detect that, or rescue it, or something. And now we’ve reached the point were there’s enough lines of code that I’d like to not have to keep replicating them throughout my site code. Actually, we’ve reached the point that my own personal code is currently at. At the moment, my processForm(formHash) code is expecting a hash with a single key that is the name of the Model object to create or update, with the value of that key being a hash of {:columnname => value. . . .} It doesn’t have all the error checking in place yet, but it’s working so far.

def processForme(formHash)
objName, objHash = formHash.to_a[0]
modelClass = eval(objName.to_s.capitalize)
objHash.delete_empty!
pkField = modelClass.primary_key
if pkValue = objHash[pkField.to_s] then
if object = modelClass[pkValue] then # it's an update
objHash.delete(pkField)
object.update(objHash)
else
object = modelClass.create(objHash)
end
end
end

What I am currently planning to do with this, is change the parameters it takes into processForm(manifest, completeFormHash). The second parameter is just everything that came back, as is. The first parameter is a manifest that is embedded in the method that’s receiving the returned data, the web page or whatever. Right now, it looks rather like this:

def savechanges
unless request.params["Cancel"] then
processForme({:sku => request.params["sku"]})
end
redirect(route(:display, {:id => request["sku"]['id']}), :port=>nil)
end

but it will hopefully end up looking more like

def savechanges
processForme([:sku], request.params)
redirect(route(:display, {:id => request["sku"]['id']}), :port=>nil)
end

I can think of a couple of different ways to have the manifest specify nested associated records; I haven’t come to any decisions on that yet. The multi-object code isn’t safe without a manifest, but this approach provides that manifest explicitly when the form returns, retaining HTML’s statelessness and not requiring cryptography. Also, this is where data transformation or pre-processing can be done. If this is a “change my password” form, ‘savechanges’ is where you can extract that field from the form data, extract the “please retype” field, confirm they’re identical, hash it, insert the hash into the form data, and then call processForme.

I’ll also probably add code to processForme to handle restricted primary keys, because I’ve tripped over that a few times. Probably I’ll figure out how to include that in the manifest: “please bypass the key restriction for this record if needed.”

[jeremyevans]

Google Group Date: Tue, 3 Jun 2014 14:23:10 -0700
Google Group Sender: [email protected]

On Tuesday, June 3, 2014 1:47:43 PM UTC-7, Dave Howell wrote:

The trivial case is Object.update(formHash); so trivial that you wouldn’t
bother normalizing it. But does that even work? Assume the hash contains
the primary key (I’m a little rushed today or I’d test it to see what
happens); will Sequel update the row referenced by the primary key or throw
an error?

No, it doesn't work, assuming that Object is a Sequel::Model class. Before
Sequel 4 (or when using the scissors plugin), that will update every single
row in your table as specified by the hash, as Model.update used to call
Dataset#update on the model's dataset.

In general, you should pass the necessary information to get the object to
update (usually the primary key, but any unique field will work) via the
URI. Don't use /update/players as the URI, use /update/players/1, and find
the appropriate instance using the URI information. In general you'll want
to do this via with_pk! or first!, so that a Sequel::NoMatchingRow
exception is raised if the matching row is not found. This is better than
using Model#[] which returns nil if the row is not found, as that generally
results in having a NoMethodError raised later.

At the moment, my processForm(formHash) code is expecting a hash with a

single key that is the name of the Model object to create or update, with
the value of that key being a hash of {:columnname => value. . . .} It
doesn’t have all the error checking in place yet, but it’s working so far.

    def processForme(formHash) 
            objName, objHash = formHash.to_a[0] 
            modelClass = eval(objName.to_s.capitalize) 

This appears to be a trivial to exploit remote code execution vulnerability.

            objHash.delete_empty! 
            pkField = modelClass.primary_key 
            if pkValue = objHash[pkField.to_s] then 
                    if object = modelClass[pkValue] then # it's an 

update

                            objHash.delete(pkField) 
                            object.update(objHash) 
                    else 
                            object = modelClass.create(objHash) 

This doesn't make sense to me. If the pkValue is provided and doesn't
match a row, instead of raising an exception (since they are trying to
update a row that doesn't exist), you create a new row, and let the user
pick the primary key. On databases that use sequences, this causes
problems as the values the user uses can be chosen by the sequence later,
resulting in uniqueness constraint violations down the road. Even if you
think you want this, it suffers from an obvious race condition if you allow
deletes, since a delete could come in between the time a form is displayed
and when it is submitted.

What I am currently planning to do with this, is change the parameters it

takes into processForm(manifest, completeFormHash). The second parameter is
just everything that came back, as is. The first parameter is a manifest
that is embedded in the method that’s receiving the returned data, the web
page or whatever. Right now, it looks rather like this:

    def savechanges 
            unless request.params["Cancel"] then 
                    processForme({:sku => request.params["sku"]}) 
            end 
            redirect(route(:display, {:id => request["sku"]['id']}), 

:port=>nil)

    end 

but it will hopefully end up looking more like

    def savechanges 
            processForme([:sku], request.params) 
            redirect(route(:display, {:id => request["sku"]['id']}), 

:port=>nil)

    end 

This is vulnerable to redirect hijacking, since request["sku"]['id'] is
controlled by the user. That's not a huge issue unless there are CSRF
vulnerabilities, but if there are, it allows arbitrary sites to trick users
with access to your site into modifying your sites database without
detection.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 3 Jun 2014 14:50:59 -0700
Google Group Sender: [email protected]

On Jun 3, 2014, at 14:23 , Jeremy Evans [email protected] wrote:

            objHash.delete_empty! 
            pkField = modelClass.primary_key 
            if pkValue = objHash[pkField.to_s] then 
                    if object = modelClass[pkValue] then # it's an update 
                            objHash.delete(pkField) 
                            object.update(objHash) 
                    else 
                            object = modelClass.create(objHash) 

This doesn't make sense to me. If the pkValue is provided and doesn't match a row, instead of raising an exception (since they are trying to update a row that doesn't exist), you create a new row, and let the user pick the primary key.

Hah. Yea, that will have to be a special case when I expand this routine. I’m using it with two forms right now, but only one of them allows me to create a new object, and the primary key for a Sku is the skucode. It would be like an ISBN: the database doesn’t get to choose that. (It used to be a UUID, but since skucodes also have to be unique, it just got to be a huge hassle to always be using the skucode for the reference and having to include “skus” for everything in order to get from the skucode to an item, or the inventory, or blah blah blah.) You’d said earlier that you thought updating and creating should be handled by separate routines. In general, I think that’d be fine, so feel free to assume the if statement is not present.

This is vulnerable to redirect hijacking, since request["sku"]['id'] is controlled by the user.

Feel free to replace that with redirect code you like. If I can figure out a reasonable way to have processForme return the objects that it updated, that would be ideal, because I could then use the ID from that as the parameter for /display. At the moment, if I tried to use this specific code with a form that created a new object (like an Item) where the database assigns the primary key value, then my redirect will fail for lack of knowledge of what the new object’s primary key is. With my older code that calls my library, I almost always used the id of the instantiated object instead of the one passed in with the form.

[jeremyevans]

Google Group Date: Tue, 3 Jun 2014 15:02:44 -0700
Google Group Sender: [email protected]

On Tuesday, June 3, 2014 2:51:03 PM UTC-7, Dave Howell wrote:

This is vulnerable to redirect hijacking, since request["sku"]['id'] is
controlled by the user.

Feel free to replace that with redirect code you like. If I can figure out
a reasonable way to have processForme return the objects that it updated,
that would be ideal, because I could then use the ID from that as the
parameter for /display. At the moment, if I tried to use this specific code
with a form that created a new object (like an Item) where the database
assigns the primary key value, then my redirect will fail for lack of
knowledge of what the new object’s primary key is. With my older code that
calls my library, I almost always used the id of the instantiated object
instead of the one passed in with the form.

It looks like your processForme method returns the object update/created,
as both Model#update and Model.create return the object, so I'm not sure
what the issue is there.

Thanks,
Jeremy

[jeremyevans]

Google Group Date: Tue, 3 Jun 2014 15:14:56 -0700
Google Group Sender: [email protected]

On Jun 3, 2014, at 15:02 , Jeremy Evans [email protected] wrote:

It looks like your processForme method returns the object update/created, as both Model#update and Model.create return the object, so I'm not sure what the issue is there.

There probably isn’t an issue. I just hadn’t looked back at it to see if that’d work yet.