Recently I was promoting a "Rails" app, that was going to be used by another division. And was asking my "boss" to login, and it got me thinking about log file "security". One of the things that has been bothering me for a while was when I was running a app, and looking at the log, either in production or development I would always "say" a few words when a user was loggin in while I was watching. One of my hero's has a favorite saying, "To much information".
To illustrate:
Processing AccountController#login (for 127.0.0.1 at 2008-01-31 13:10:58) [POST]
Session ID: 9a85051d00c5aee3dc20305ac0dd315e
Parameters: {"commit"=>"Log in", "action"=>"login", "controller"=>"adminspace/account", "login"=>"user", "password"=>"secret"}
And you wll see this in both production and development.
No there was a few solutions when I was googling:
1. Turn the logging to warning
in config/environments/production.rb
# Fix the logging so sensitive information is not logged
config.log_level = :warn
Cons: You loose all your request, now you log analyser and you have very little information on what is going on in your application.
2. "Filtering"
Now this is "nice". And is good even in development side. (ie I use ldap based authentication, so in development the account and password is REAL). So it
lets me still see whats going on, without printing out the password (My password)
1000 plus times.
The fix:
in app/controllers/application.rb
filter_parameter_logging "password"
This will get rid of any filtered parameter.
The result (in the development log):
Processing AccountController#login (for 127.0.0.1 at 2008-01-31 13:10:58) [POST]
Session ID: 9a85051d00c5aee3dc20305ac0dd315e
Parameters: {"commit"=>"Log in", "action"=>"login", "controller"=>"adminspace/account", "login"=>"user", "password"=>"[FILTERED]"}
Lovely.
You can also hide other things as well if you want to keep them secure, credit card numbers etc are good examples.
I tend to think this should be your "default" setting in any application.
It will be in mine from now on.
This works in 2.0.2 rails very nicely. (It was actually added quite a while back)
If you ask me it should be the "default" in the template when generating a rails applicaton.
Wednesday, January 30, 2008
Sunday, January 27, 2008
Oracle and Rails - Moving tables to new tablespaces
Hmmm. Things are growing in my oracle tables, and I really want to
move things around a bit. So moving a "application" or "user" to a new
tablespace is resonably common.
For me, language for first resort is ruby, so I have quite
a few "little" scripts to do oracle tasks.
Here's today's, move all of a users tables to a new tablespace.
require 'DBI'
require 'pp'
#movetablepace.rb
#***************************************
#make a Oracle Connection
fromdbconstr = ARGV[0] # For example - DBI:OCI8://myoracleserver/oracleinstance
fromdbconuser = ARGV[1] # MYRAILSDBUSER-DEVELOP
fromdbconpass = ARGV[2] # MYRAILSDBUSER-PASSWORD
dataspacename = ARGV[3] # APPLICATION-DATASPACE
fromdb = DBI.connect(fromdbconstr, fromdbconuser, fromdbconpass)
tables = fromdb.tables
tables.each do |tablename|
tablename = tablename.strip
if tablename.length == 0
next
end
if tablename.length > 4
if tablename[0,4] == "BIN$"
next
end
end
puts "Move Table [#{tablename}] to new tablespace"
sqlmove = "alter table " + '"' + tablename + '" move tablespace "' + dataspacename + '"'
puts sqlmove
begin
result = fromdb.execute(sqlmove)
rescue
puts "Table [#{tablename}] not moved"
end
end
And after you do that, you need to "rebuild" the indexes into the new tablespace
require 'DBI'
require 'pp'
#make a Oracle Connection
fromdbconstr = ARGV[0] # For example - DBI:OCI8://myoracleserver/oracleinstance
fromdbconuser = ARGV[1] # MYRAILSDBUSER-DEVELOP
fromdbconpass = ARGV[2] # MYRAILSDBUSER-PASSWORD
dataspacename = ARGV[3] # APPLICATION-DATASPACE
fromdb = DBI.connect(fromdbconstr, fromdbconuser, fromdbconpass)
tables = fromdb.tables
sqlget = "select index_name from user_indexes"
indexs = fromdb.execute(sqlget)
indexs.each do |theindex|
indexname = theindex[0]
printf "Rebuilding Index %s\n", indexname
sqlrebuild = 'alter index "' + indexname + '" rebuild tablespace "' + dataspacename + '"'
begin
result = fromdb.execute(sqlrebuild)
rescue
printf "[%s] Index not rebult\n", indexname
end
end
move things around a bit. So moving a "application" or "user" to a new
tablespace is resonably common.
For me, language for first resort is ruby, so I have quite
a few "little" scripts to do oracle tasks.
Here's today's, move all of a users tables to a new tablespace.
require 'DBI'
require 'pp'
#movetablepace.rb
#***************************************
#make a Oracle Connection
fromdbconstr = ARGV[0] # For example - DBI:OCI8://myoracleserver/oracleinstance
fromdbconuser = ARGV[1] # MYRAILSDBUSER-DEVELOP
fromdbconpass = ARGV[2] # MYRAILSDBUSER-PASSWORD
dataspacename = ARGV[3] # APPLICATION-DATASPACE
fromdb = DBI.connect(fromdbconstr, fromdbconuser, fromdbconpass)
tables = fromdb.tables
tables.each do |tablename|
tablename = tablename.strip
if tablename.length == 0
next
end
if tablename.length > 4
if tablename[0,4] == "BIN$"
next
end
end
puts "Move Table [#{tablename}] to new tablespace"
sqlmove = "alter table " + '"' + tablename + '" move tablespace "' + dataspacename + '"'
puts sqlmove
begin
result = fromdb.execute(sqlmove)
rescue
puts "Table [#{tablename}] not moved"
end
end
And after you do that, you need to "rebuild" the indexes into the new tablespace
require 'DBI'
require 'pp'
#make a Oracle Connection
fromdbconstr = ARGV[0] # For example - DBI:OCI8://myoracleserver/oracleinstance
fromdbconuser = ARGV[1] # MYRAILSDBUSER-DEVELOP
fromdbconpass = ARGV[2] # MYRAILSDBUSER-PASSWORD
dataspacename = ARGV[3] # APPLICATION-DATASPACE
fromdb = DBI.connect(fromdbconstr, fromdbconuser, fromdbconpass)
tables = fromdb.tables
sqlget = "select index_name from user_indexes"
indexs = fromdb.execute(sqlget)
indexs.each do |theindex|
indexname = theindex[0]
printf "Rebuilding Index %s\n", indexname
sqlrebuild = 'alter index "' + indexname + '" rebuild tablespace "' + dataspacename + '"'
begin
result = fromdb.execute(sqlrebuild)
rescue
printf "[%s] Index not rebult\n", indexname
end
end
Friday, January 25, 2008
When the C-LEVEL(CTO,CEO,CFO) ask so why should we use Rails
Ok Mr. Big CTO, why shuld we use RAILS?
A question that was recently asked in the Ruby forums.
So having done a few C-Level jobs, I thought I'd answer.
1. Is it going to be around a long time - Yes - Ruby itself has, and
Rails has hit 2. Plus recognition from
major software firms. (Oracle offered class on it on the Shanghai
Training Session,), oracle has support.
Major media companies are using it. Other MNC are using. Trading(As in
stock, on wallstreet) are using it.
Look at the "books" available. Need one every week. Look at the number
of web articles. By every metric its
everywhere.
2. Is training available? Yes, on multiple fronts. Also fresh java
grads tend to pick it up easier.
3. Does it marry with my "enterprise" systems. Now this one is
interesting. Ruby can marry to just
about anything, so thus "rails" can. I am just finishing a "marriage"
between Unify and Rails. The original
applicaiton is over 15 years old, and the data is still worth
millions. Of course I "moved" the data over to
oracle. (With ruby). I've also did "bridges" between applications.
3. Is it secure? If its running on "Linux" would be my answer. Look at
the security guide. Put a https proxy(Nginx)
in fromt. 100x more secure than windows.
4. Is it scalable. Big question. But in a short answer, yes yes yes.
For our users, 3000-4000 is max user count.
I even was helping a friend with handling million plus user bases.
Hardware is cheap, with nginx, mongrel clusters, and
good hardware its falling off a log. (If your public hosting, even a
single server can handle your average load").
5. Can I get consultants? Yes. Look at the tavnav guys for one. There
great.
6. But does it take "long" to do the job?
Its the "fastest" system I've found to get thing done fast, neat, and quick.
Look up "DRY" under Rails. It stands for "Dont Repeat Yourself".
7. Does it do "Windows"
Yes very well. How about a "Applicatoin-to-Application" Bridge done
in a week as a Windows "Service"
8. Does it do "Clusters"
Yes
9. Does it do "Linux"
Yes
10. Is it better than Java
Yes - very. So much more concise.
A question that was recently asked in the Ruby forums.
So having done a few C-Level jobs, I thought I'd answer.
1. Is it going to be around a long time - Yes - Ruby itself has, and
Rails has hit 2. Plus recognition from
major software firms. (Oracle offered class on it on the Shanghai
Training Session,), oracle has support.
Major media companies are using it. Other MNC are using. Trading(As in
stock, on wallstreet) are using it.
Look at the "books" available. Need one every week. Look at the number
of web articles. By every metric its
everywhere.
2. Is training available? Yes, on multiple fronts. Also fresh java
grads tend to pick it up easier.
3. Does it marry with my "enterprise" systems. Now this one is
interesting. Ruby can marry to just
about anything, so thus "rails" can. I am just finishing a "marriage"
between Unify and Rails. The original
applicaiton is over 15 years old, and the data is still worth
millions. Of course I "moved" the data over to
oracle. (With ruby). I've also did "bridges" between applications.
3. Is it secure? If its running on "Linux" would be my answer. Look at
the security guide. Put a https proxy(Nginx)
in fromt. 100x more secure than windows.
4. Is it scalable. Big question. But in a short answer, yes yes yes.
For our users, 3000-4000 is max user count.
I even was helping a friend with handling million plus user bases.
Hardware is cheap, with nginx, mongrel clusters, and
good hardware its falling off a log. (If your public hosting, even a
single server can handle your average load").
5. Can I get consultants? Yes. Look at the tavnav guys for one. There
great.
6. But does it take "long" to do the job?
Its the "fastest" system I've found to get thing done fast, neat, and quick.
Look up "DRY" under Rails. It stands for "Dont Repeat Yourself".
7. Does it do "Windows"
Yes very well. How about a "Applicatoin-to-Application" Bridge done
in a week as a Windows "Service"
8. Does it do "Clusters"
Yes
9. Does it do "Linux"
Yes
10. Is it better than Java
Yes - very. So much more concise.
Tuesday, January 22, 2008
Advanced Rails Migrations - Adding and Updating Related Table ID's
Ever have that "rails" problem.
You imported a butch of "existing" tables to create a rails application,
and you dont have appropriate index/id's to help with your belong_to, and
associated tables?
Well Migrations can do it for you.
Example migration using two tables that are in ActiveRecord.
This one builds a link between a inventory table, and a customer table.
class UpdateXco < ActiveRecord::Migration
def self.up
add_column :REG_INV_SERS, :REG_CUST_ID, :integer
RegInvSer.reset_column_information
RegInvSer.find(:all).each do |record|
regcustomer = RegCust.find(:first, :conditions => "CUST_NBR = '#{record.cust_nbr}'")
if not regcustomer.nil?
record.reg_cust_id = regcustomer.id
record.save
end
end
end
def self.down
remove_column "REG_INV_SERS", "REG_CUST_ID"
end
end
Another one, that illustrates doing a table import from a dbase/foxpro table, with a id update as well.
class Updatesalesorder < ActiveRecord::Migration
def self.up
thepathname = "h:xxxxx.DBF"
table = DBF::Table.new(thepathname)
table.records.each do |record|
mysalesorder = salesorder.create(record.attributes)
mycustomer = customer.find(:first, :conditions => "custno = '#{record.custno}'")
if not mycustomer.nil?
mysalesorder.customer_id = mycustomer.id
mysalesorder.save
end
end
end
def self.down
drop_table "salesorders"
end
end
You imported a butch of "existing" tables to create a rails application,
and you dont have appropriate index/id's to help with your belong_to, and
associated tables?
Well Migrations can do it for you.
Example migration using two tables that are in ActiveRecord.
This one builds a link between a inventory table, and a customer table.
class UpdateXco < ActiveRecord::Migration
def self.up
add_column :REG_INV_SERS, :REG_CUST_ID, :integer
RegInvSer.reset_column_information
RegInvSer.find(:all).each do |record|
regcustomer = RegCust.find(:first, :conditions => "CUST_NBR = '#{record.cust_nbr}'")
if not regcustomer.nil?
record.reg_cust_id = regcustomer.id
record.save
end
end
end
def self.down
remove_column "REG_INV_SERS", "REG_CUST_ID"
end
end
Another one, that illustrates doing a table import from a dbase/foxpro table, with a id update as well.
class Updatesalesorder < ActiveRecord::Migration
def self.up
thepathname = "h:xxxxx.DBF"
table = DBF::Table.new(thepathname)
table.records.each do |record|
mysalesorder = salesorder.create(record.attributes)
mycustomer = customer.find(:first, :conditions => "custno = '#{record.custno}'")
if not mycustomer.nil?
mysalesorder.customer_id = mycustomer.id
mysalesorder.save
end
end
end
def self.down
drop_table "salesorders"
end
end
Monday, January 21, 2008
Ruby - A "class" tutorial using a Unify DBI interface as a example
This is a "good" example of a "class".
It implemnets a simple "dbi" like interface to a unify IDS database.
(This is a embedded database used in linux, and embedded devices)
Recently needed to access data from a unify database in "rails".
So of course, needed a "dbi" like layer to work with one of my scripts to
pull the data. (See previous posts on data conversion and migrations).
So it was natural to create a "class". A class is a "object" consisting of code,
or methods, and "data". The data can come in two types "instance" and class data.
Here everything is "instance" data. So when we create a new "Unify" object, we
get another set of "data" for that instance.
In our example, we use two temp files per instance to communication to Unify sql command via a "shell" command. One is for the SQL statement, and one is for the "results".
Also the lovely thing about "instances" is it makes it easy to see all your "instance" data, simply by doing a "pp" (Pretty Print) on the "instance" variable.
So:
mydb = Unify.new(database,user,password)
pp mydb (Will let you see what going on with the instance)
Note that all the "@" instance varibles are recreated on each "new".
This allows this simple example to have multiple outstanding "queries", without
getting confused.
class Unify
def make_tmpname(type,instance)
return 'unify-' + type + '-' + instance + '.tmp'
end
def execute_unify_sql(thesql)
thequery = File.open(@uqueryfilename,"w")
thequery.puts "lines 0"
# thequery.puts "separator ','
thequery.puts thesql
thequery.close
thecommand = 'SQL ' + @uqueryfilename + '> ' + @udatafilename
result = system(thecommand)
if result == false
sleep(10)
result = system(thecommand)
end
theresult = File.open(@udatafilename)
@udatafile = theresult
return theresult
end
def initialize(database,user,password)
@udatabase = database
@uuser = user
@upassword = password
@udatafile = nil
@udatafilename = nil
@uqueryfilename = nil
queryinstance = rand(10000).to_s
@udatafilename = make_tmpname('result',queryinstance)
@uqueryfilename = make_tmpname('query',queryinstance)
end
def fields(tablename)
thesql = 'fields ' + tablename
thedata = execute_unify_sql(thesql)
header = thedata.gets
header = thedata.gets #throw away the two header lines
result = Array.new
thedata.readlines.each { |line|
theline = line.chomp!
# theline <= '|' # last element does not terminate with |
element = theline.split
result << element
}
self.finish()
return result
end
def tables
thedata = execute_unify_sql("tables")
thecontents = thedata.read
self.finish
mytables = thecontents.split
return mytables
end
def execute(sql)
#For execute create a new "instance"
unifyinstance = Unify.new(@udatabase,@uuser,@upassword)
if sql[-1,1] == ';'
sql[-1,1] = '/'
end
theresultfile = unifyinstance.execute_unify_sql(sql)
return unifyinstance
end
def fetch()
@udatafile.pos = 0 #Make sure we are at the beginning
thetable = Array.new
while thedata = @udatafile.gets
thedata.chomp!
therow = thedata.split('|',-1) # No supressed fields
thetable << therow
end
return thetable
end
def finish()
@udatafile.close
File.delete(@udatafilename)
File.delete(@uqueryfilename)
end
end
It implemnets a simple "dbi" like interface to a unify IDS database.
(This is a embedded database used in linux, and embedded devices)
Recently needed to access data from a unify database in "rails".
So of course, needed a "dbi" like layer to work with one of my scripts to
pull the data. (See previous posts on data conversion and migrations).
So it was natural to create a "class". A class is a "object" consisting of code,
or methods, and "data". The data can come in two types "instance" and class data.
Here everything is "instance" data. So when we create a new "Unify" object, we
get another set of "data" for that instance.
In our example, we use two temp files per instance to communication to Unify sql command via a "shell" command. One is for the SQL statement, and one is for the "results".
Also the lovely thing about "instances" is it makes it easy to see all your "instance" data, simply by doing a "pp" (Pretty Print) on the "instance" variable.
So:
mydb = Unify.new(database,user,password)
pp mydb (Will let you see what going on with the instance)
Note that all the "@" instance varibles are recreated on each "new".
This allows this simple example to have multiple outstanding "queries", without
getting confused.
class Unify
def make_tmpname(type,instance)
return 'unify-' + type + '-' + instance + '.tmp'
end
def execute_unify_sql(thesql)
thequery = File.open(@uqueryfilename,"w")
thequery.puts "lines 0"
# thequery.puts "separator ','
thequery.puts thesql
thequery.close
thecommand = 'SQL ' + @uqueryfilename + '> ' + @udatafilename
result = system(thecommand)
if result == false
sleep(10)
result = system(thecommand)
end
theresult = File.open(@udatafilename)
@udatafile = theresult
return theresult
end
def initialize(database,user,password)
@udatabase = database
@uuser = user
@upassword = password
@udatafile = nil
@udatafilename = nil
@uqueryfilename = nil
queryinstance = rand(10000).to_s
@udatafilename = make_tmpname('result',queryinstance)
@uqueryfilename = make_tmpname('query',queryinstance)
end
def fields(tablename)
thesql = 'fields ' + tablename
thedata = execute_unify_sql(thesql)
header = thedata.gets
header = thedata.gets #throw away the two header lines
result = Array.new
thedata.readlines.each { |line|
theline = line.chomp!
# theline <= '|' # last element does not terminate with |
element = theline.split
result << element
}
self.finish()
return result
end
def tables
thedata = execute_unify_sql("tables")
thecontents = thedata.read
self.finish
mytables = thecontents.split
return mytables
end
def execute(sql)
#For execute create a new "instance"
unifyinstance = Unify.new(@udatabase,@uuser,@upassword)
if sql[-1,1] == ';'
sql[-1,1] = '/'
end
theresultfile = unifyinstance.execute_unify_sql(sql)
return unifyinstance
end
def fetch()
@udatafile.pos = 0 #Make sure we are at the beginning
thetable = Array.new
while thedata = @udatafile.gets
thedata.chomp!
therow = thedata.split('|',-1) # No supressed fields
thetable << therow
end
return thetable
end
def finish()
@udatafile.close
File.delete(@udatafilename)
File.delete(@uqueryfilename)
end
end
Rails 2.0.2 Issues and Problems Resolved
Now this was fun, my main Rails box is now running Rails 2.0.2
First problem, when running Rake task, I get complaints about Active Recrd
C:\projects\ror\tempco>rake extract_fixtures
(in C:/projects/ror/tempco)
rake aborted!
can't activate activerecord (= 1.15.3), already activated activerecord-2.0.2]
This one stumped me for a moment, but then realized that the "old" gems may
be confusing issues. So a little cleanup is in order.
gem cleanup
This command cleans up all "the" old gems.
And solved my problems
The next problem I had, was:
C:\projects\ror\tempco>rake extract_fixtures
(in C:/projects/ror/tempco)
rake aborted!
Please install the oracle adapter: `gem install activerecord-oracle-adapter` (no
such file to load -- active_record/connection_adapters/oracle_adapter)
Rails 2 does not by default install ActiveRecord adapter for oracle.
Of course if you try the gem install command the error message recommends,
it still does not give you gratification.
The proper command that works is:
C:\projects\ror\tempco>gem install activerecord-oracle-adapter
--source http://gems.rubyonrails.org
Bulk updating Gem source index for: http://gems.rubyonrails.org
Successfully installed activerecord-oracle-adapter-1.0.0
1 gem installed
And now, I try to do a rake db:schema:dump
And the next problem occurs, seems like there is a "bug" in the
oracle adapter.
The fix seems to be here:
http://dev.rubyonrails.org/attachment/ticket/9062/fix_oracleadapter_problem_indexes.diff
But does not match the code in the 1.0 adapter. Hmmm.
Ok, lets try to setup a "session" migration.
Hmm same error
C:\projects\ror\tempco>
C:\projects\ror\tempco>rake db:migrate
(in C:/projects/ror/tempco)
== 1 CreateSessions: migrating ================================================
-- create_table(:sessions)
-> 38.5940s
-- add_index(:sessions, :session_id)
-> 0.0930s
-- add_index(:sessions, :updated_at)
-> 0.0630s
== 1 CreateSessions: migrated (38.7500s) ======================================
rake aborted!
select_rows is an abstract method
Ok, do a bit more google work, and what do we find:
http://dev.rubyonrails.org/ticket/10415
Insert the code into the oracle adapter, and now no error on our migration.
And our db:schema:dump works properly.
First problem, when running Rake task, I get complaints about Active Recrd
C:\projects\ror\tempco>rake extract_fixtures
(in C:/projects/ror/tempco)
rake aborted!
can't activate activerecord (= 1.15.3), already activated activerecord-2.0.2]
This one stumped me for a moment, but then realized that the "old" gems may
be confusing issues. So a little cleanup is in order.
gem cleanup
This command cleans up all "the" old gems.
And solved my problems
The next problem I had, was:
C:\projects\ror\tempco>rake extract_fixtures
(in C:/projects/ror/tempco)
rake aborted!
Please install the oracle adapter: `gem install activerecord-oracle-adapter` (no
such file to load -- active_record/connection_adapters/oracle_adapter)
Rails 2 does not by default install ActiveRecord adapter for oracle.
Of course if you try the gem install command the error message recommends,
it still does not give you gratification.
The proper command that works is:
C:\projects\ror\tempco>gem install activerecord-oracle-adapter
--source http://gems.rubyonrails.org
Bulk updating Gem source index for: http://gems.rubyonrails.org
Successfully installed activerecord-oracle-adapter-1.0.0
1 gem installed
And now, I try to do a rake db:schema:dump
And the next problem occurs, seems like there is a "bug" in the
oracle adapter.
The fix seems to be here:
http://dev.rubyonrails.org/attachment/ticket/9062/fix_oracleadapter_problem_indexes.diff
But does not match the code in the 1.0 adapter. Hmmm.
Ok, lets try to setup a "session" migration.
Hmm same error
C:\projects\ror\tempco>
C:\projects\ror\tempco>rake db:migrate
(in C:/projects/ror/tempco)
== 1 CreateSessions: migrating ================================================
-- create_table(:sessions)
-> 38.5940s
-- add_index(:sessions, :session_id)
-> 0.0930s
-- add_index(:sessions, :updated_at)
-> 0.0630s
== 1 CreateSessions: migrated (38.7500s) ======================================
rake aborted!
select_rows is an abstract method
Ok, do a bit more google work, and what do we find:
http://dev.rubyonrails.org/ticket/10415
Insert the code into the oracle adapter, and now no error on our migration.
And our db:schema:dump works properly.
Subscribe to:
Posts (Atom)