Halaman

Rabu, 04 Januari 2012

Verifying numbers with the Luhn algorithm

The Luhn algorithm also known as the "modulus 10" algorithm, is a checksum formula used to verifying a variety of identification numbers, such as credit card numbers and IMEI numbers.

Luhn algorithm provide simple and inexpensive method for computing check digits and provide a simple device for verifying numbers which have a single check digit appended. The Luhn Mod-10 Method is used in a checking system for multi-digit numbers to indicate whether, in transmitting a number, an error has been made, such as a transposition of the digits. It may be used where a great many parts are ordered, manufactured, invoiced, shipped,and billed by multi-digit numbers.

When a number is first assigned to a new part a check digit is computed. This single check digit is appended to the righthand of the part number. The value of this check digit is so computed that in verifying the number by cross addition of multiple digits of the number and the check digit, in accordance with a rule of substitution, the result will be a zero.

e.g: The multi-digit number is 3046016202394 and the check digit is 9. The whole number is 30460162023949.

3 0 4 6 0 1 6 2 0 2 3 9 4 9
x2 x1 x2 x1 x2 x1 x2 x1 x2 x1 x2 x1 x2 x1
-- -- -- -- -- -- -- -- -- -- -- -- -- --
6 0 8 6 0 1 12 2 0 2 6 9 8 9
6+ 0+ 8+ 6+ 0+ 1+ 3+ 2+ 0+ 2+ 6+ 9+ 8+ 9 = 60

In the above summation, any two-digit product is included as the sum of its two digits. In that case the sum of number 12 is 3 (1 + 2). Since the sum of the digits in the bottom row is 60, which is divisible by 10, so the Number is valid because the 60/10 yields no remainder, or 60 mod 10=0. The computation PASSED since the result is zero.

Here I use PostgreSQL stored function to do generate a check digit for string of numbers.

CREATE OR REPLACE FUNCTION gen_lun10_checkdigit(character varying)
RETURNS character AS
$BODY$
SELECT ((10 - SUM(doubled_digit / 10 + doubled_digit % 10) % 10) % 10)::character
FROM (SELECT MOD( ($1::int8 / (10^n)::int8), 10::int8 ) * (2 - MOD(n,2)) AS doubled_digit
FROM generate_series(0, length($1)- 1) AS n) AS doubled_digits ;
$BODY$
LANGUAGE sql IMMUTABLE STRICT
COST 100;

To find the check digit, call the function by sending a multi-digit number, e.g:


select gen_lun10_checkdigit('3046016202394') as checkdigit;
checkdigit
----------
9

In real world, the Lun algorithm is widely used to verifying numbers such Credit Card and IMEI numbers.

To work with these I wrote this function in PostgreSQL.

CREATE OR REPLACE FUNCTION gen_lun10_number(pprefix character varying,pnumextralen integer) RETURNS character varying AS
$BODY$
DECLARE
vfmt character varying(16);
vnumber bigint;
vnum1 bigint;
vnum2 bigint;
x text;
vresult character varying(19);
BEGIN
pnumextralen := pnumextralen - 1;
vfmt := REPEAT('9', 16);
vnum1 := REPEAT('9', pnumextralen)::int8;
vnum2 := REPEAT('1', pnumextralen)::int8;

SELECT TRUNC (random() * ( vnum2 - vnum1) + vnum1) INTO vnumber;
x = pprefix || vnumber :: text;

vnumber = x::int8;

SELECT 10 * vnumber + ((10 - SUM(doubled_digit / 10 + doubled_digit % 10) % 10) % 10)::int8 FROM (SELECT MOD( (vnumber/ (10^n) ::int8), 10) * (2 - MOD(n,2))::int8 AS doubled_digit FROM generate_series(0, LENGTH(x)-1) AS n) AS doubled_digits INTO vresult;

IF (substr (pprefix,1,1)='0') THEN
vresult := '0' || vresult;
END IF;
RETURN vresult;
END;
$BODY$
LANGUAGE plpgsql VOLATILE COST 100;



You could also find PostgreSQL functions here or other languages in here.


A. Bank Card Number
A bank card number is the primary account number found on credit cards and bank cards. It has a certain amount of internal structure and shares a common numbering scheme. Credit card
numbers are a special case of ISO/IEC 7812 bank card numbers.

An ISO/IEC 7812 number is typically 16 digits in length. It consists of:
  • a six-digit Issuer Identification Number (IIN), the first digit of which is the Major Industry Identifier (MII),
  • a variable length (up to 12 digits) individual account identifier,
  • a single check digit calculated using the Luhn algorithm.

Using the above function, given prefix and length, we got these random number, and I use this site to check credit card.

AMEX
select gen_lun10_number('37',13);--->379385470967314
select gen_lun10_number('34',13);--->344631823379950

VISA
select gen_lun10_number('4539',12); --->4539442682130873
select gen_lun10_number('4556',12);--->4556142088753892

DISCOVER
select gen_lun10_number('6011',12);-->6011865781139571
select gen_lun10_number('622126',10);-->6221269895829522

DINNERS
select gen_lun10_number('304',11);--->30460162023949
select gen_lun10_number('305',11);--->30570718114024

To generate 10 number with prefix '12345' with length of appended numbers 11, simply I use this call.

SELECT c.result
FROM (SELECT gen_lun10_number('12345',11) AS result
FROM generate_series(0,9)) as c;

result
-----------------
1234517933942749
1234569174718767
1234582260609360
1234579067268580
1234516226418235
1234565468996135
1234535863889839
1234535902922765
1234594959690825
1234569877669242



B. IMEI
The International Mobile Equipment Identity or IMEI is a number, usually unique, to identify GSM, WCDMA, and iDEN mobile phones, as well as some satellite phones. It is usually found printed inside the
battery compartment of the phone. The IMEI number is used by the GSM network to identify valid devices and therefore can be used for stopping a stolen phone from accessing the network in that country.

For example, if a mobile phone is stolen, the owner can call his or her network provider and instruct them to "blacklist" the phone using its IMEI number. This renders the phone useless on that network and sometimes other networks too, whether or not the phone's SIM is changed.

The IMEI is only used for identifying the device and has no permanent or semi-permanent relation to the subscriber. Instead, the subscriber is identified by transmission of an IMSI number, which is stored on a SIM card that can (in theory) be transferred to any handset. However, many network and security features are enabled by knowing the current device being used by a subscriber.

To work with IMEI, we have to know IMEI Structure, IMEI number has 15 decimal digits. Actualy it has 14 digits plus a check digit. First 8 digits of IMEI number are Type Allocation Code which will give you the mobile phone brand and model. Other 7 digits are defined by manufacturer (6 are serial number and 1 is check digit).

To view IMEI of your mobile phone, just press: *#06# and press Call button.

Below were some generated
IMEI numbers with predetermined prefix and length.

Samsung SGH-A800
select gen_lun10_number('35357800',7);-->353578007008467 , 353578007478082

RiM BlackBerry Bold 9700
select gen_lun10_number('35425504',7);--> 354255042721045 , 354255045759083

Apple iPhone 3G model MB496RS
select en_lun10_number('01174400',7); -->011744003170479 , 011744001398676

Nokia 6210
select gen_lun10_number('449337',9); -->449337138770594 , 449337951410641

These sites give IMEI check and information here or here.

C. Highlights
The Luhn algorithm offers error detection, not offer security, just like a CRC in software. The algorithm help us from accidental errors. By applying the Luhn algorithm using PostgreSQL, I could generate identification numbers for data verification and validation.

You probably interest to read this article that gives inspiration, ...an internal auditor discovered the Luhn algorithm, a simple redundancy check that can be used to validate different identification numbers during fraud and IT audit investigations.

Selasa, 03 Januari 2012

Data Verification and Validation

Verification is a quality control process that is used to evaluate whether a product, service, or system complies with regulations, specifications, or conditions imposed at the start of a development phase. Verification can be in development, scale-up, or production. This is often an internal process.

Validation is a quality assurance process of establishing evidence that provides a high degree of assurance that a product, service, or system accomplishes its intended requirements. This often involves acceptance of fitness for purpose with end users and other product stakeholders. This is often an external process.

It is sometimes said that validation can be expressed by the query "Are you building the right thing?" and verification by "Are you building it right?"

"Building the right thing" refers back to the user's needs, while "building it right" checks that the specifications are correctly implemented by the system. In some contexts, it is required to have written requirements for both as well as formal procedures or protocols for determining compliance.


Verification and validation is done to make sure that the data is accurate and error free as the incorrect data can lead to data deformation or miss understanding of the document.

Data verification checks that the document meets specifications and that it fulfills its intended purpose this can be done by doing some checks like double entry so that the data must be entered twice or proof reading to make sure it is accurate and no errors in it, as for validations , it can be done by coding which means giving a code to specific words so that it would be easier to enter and the probability of mistake will be less ,also format check can be done as it checks that the data are in a specific format, added to that ,spelling and grammar checks for checking the language and writing mistakes.

Data validation checks that data are valid, sensible, reasonable before they are processed it is a quality assurance process of establishing evidence that provides a high degree of assurance that the document accomplishes its intended requirements.

Selasa, 16 Agustus 2011

ALERT - configured POST variable limit exceeded

This afternoon my phone rings, the caller said that the data is already inputted was not saved.
I did a verbal check on him and because he is so confident with the situation, I said that I would check the system.

I check the server and see the error log.
# tail-f /var/log/httpd-error.log

[Tue Aug 16 12:15:12 2011] [error] [client xxx.xxx.xxx.xxx] ALERT - configured POST variable limit exceeded - dropped variable 'item_desc' (attacker 'xxx.xxx.xxx.xxx', file '/usr/ local / www / MyHome / myprg.php'), referer: http://mywww.com/myhome/myprg.php?pk=921664e4a696167707570706664466830332b4a4a2b384330324a6c75473438

It's definitely about the use of a variable that exceeds the limitations set.
After seeing this (I use PHP with Suhosin-Patch), I change configuration file /usr/local/etc/php.ini and add these two lines

suhosin.post.max_vars = 400
suhosin.request.max_vars = 400

then restart apache.

# /usr/local/sbin/apachectl stop
# /usr/local/sbin/apachectl graceful

then it works.

Thanks world!

Sabtu, 12 Februari 2011

Generate SQL Script File from Xbase

I have a task to write a new system of friend of mine. His existing system was run on Xbase, a MS DOS based application.

One of interesting part is doing data transformation from DBase format (table) into a database server such MySQL or PostgreSQL.
The scenario I planned were:
1. Read DBase table format and data
2. Generate a sql script file for creating table an inserting data
3. Run the script

Okay, there are many GUI tools already. I've a try using GUI tools such as Exportizer even another tool Foxpro-Postgresql import-export or MySQL Front. Those are good, later I arrived at dbf2sql. I compile and test the program . It works.

After some testing I realize that there are no NULL keyword generated for numeric, date, and character data type. I modify the source to generate a NULL keyword for non existing value of every field/column and generate some commands to satisfy my requirements.

Here the snippet code using C:
...
for(X = 0; X <>Field[Y].Length; X++)
{
temp[X] = *(data++);
}
temp[X] = '\0';

if(DBase->Field[Y].Type == 'N' DBase->Field[Y].Type == 'n')
{
if(!Trim) //it's a number, trim anyway...
TrimSpaces(temp, DBase->Field[Y].Length);
if(temp[0] == '.')
fprintf(output, "0%s", temp);
else
if(temp[0] == '-' && temp[1] == '.')
fprintf(output, "-0%s", &(temp[1]));
else if(temp[0] == '\0' )
fprintf(output, "null");
else
fprintf(output, "%s", temp);
}
....


The command to generate a script file:
C:\TEST>dbf2sql -i rekening.dbf -t tova.rek0 -o cmd.sql

The parameter provide input file name i.e rekening.dbf, and specify the schema and table name of the generated sql, and specify the destination to file cmd.sql. The contents of cmd.sql file cotains such following code:

DROP TABLE tova.rek0;
create table tova.rek0 (I've cut the detail);
INSERT INTO tova.rek0 VALUES ('11010001', 20, 20, 20, 4069, 4069, null, null, null, null, null, null, null, null, null, null, 0, '!', 'IB', 'L', 18450, 0, '2009-01-19', '2008-12-16', 'A', 'A', 20, 20, 0, '2009-01-19', null, null, null, null, null, null, null, null, null, null, null, 20, null, null, null, null, null, null, null, null, 0, 0, 0, 0, null, null, null, null, null, null, null, null, 0, 0, 0, 1000, null, null, null, null, null, null, null, null, null, null, null, 2000, null, null, null, null, null, null, null, null, null, null, null, 0, null, null, null, null, null, null, null, null, null, null, null, 15450, null, null, null, null, null, null, null, null, null, null, null, 'A', null, null, null, null, null, null, null, null, null, null, null, 'IB', null, null, null, null, null, null, null, null, null, null, null, 'L', null, null, null, null, null, null, null, null, null, null, null, '!', null, null, null, null, null, null, null, null, null, null, null, 'L', 'IB', 0, 'ZAMAG', null, null, null, null, null, null, null, null, null, null, null, 19, null, null, null, null, null, null, null, null, null, null, null, 0, 0, 0, 'A', 'A', 'A');

I love to use the dbf2sql to generate sql script for simplicity, customizable and speed reasons. The next step is to run the script into database. For PostgreSQL try this.

Importing SQL Script file into Postgresql table

To import a table from sql script file do this:
1. Generate a sql script file.
We could have a sql script file from many tools.

2. Run the script and specify parameter for user, database, and the sql script file, e.g:
C:\TEST>psql -Uferrari -d mydb -f cmd.sql

If the size of cmd.sql is small, we could use pgAdmin tools. If it was a large file, it will take longer to process e.g: while inserting into table. or an error will raise with ERROR: out of memory; SQL state: 53200; Detail: Failed on request of size 1048576.

Senin, 24 Januari 2011

Inserting or Changing Primary Key

SELECT script is easy to be obtained from PostgreSQL tables by using the GUI such as pgAdmin. By right-clicking on a table and then select the SELECT script, we get the script. Another way could be affordable by the coding.

First, create a table.
CREATE TABLE coba
(
pk character(12) NOT NULL,
nourut smallint NOT NULL,
nourutku smallint NOT NULL,
unit_id character(3) NOT NULL,
CONSTRAINT coba_pkey PRIMARY KEY (pk)
)
WITH (OIDS=FALSE);
ALTER TABLE coba OWNER TO binsar;


List the table's fields:
SELECT column_name FROM information_schema.columns WHERE table_name = 'coba'

Output:
"pk"
"nourut"
"nourutku"
"unit_id"


Another way...using coding
SELECT script can be generated through the following script:

SELECT ' SELECT ' || array_to_string(ARRAY(SELECT '' ||column_name
FROM information_schema.columns
WHERE table_name = 'coba'), ',') || ' FROM binsar.coba;';

Output:
SELECT pk,nourut,nourutku,unit_id FROM binsar.coba;


We may face the need to change the primary key or adding a new primary key based on the existing rows at the same table. The problems will be encountered, especially if there are many relationships between tables. For example, how do we simply insert some rows and only change the primary key value?

insert into binsar.coba VALUES ('1',1, 1,'1');
insert into binsar.coba SELECT '2',2, 2,'2';
insert into binsar.coba SELECT '3', nourut, nourutku, unit_id FROM binsar.coba WHERE pk='1';


The last command simply solve the need....certainly for a table with some fields.
If I have many tables and fields, ( think about fields name: new, dropped or modified ),
I was not diligent enough to do it. Were you?

Here the script:

SELECT 'SELECT ''300'',' || array_to_string(ARRAY(SELECT '' || column_name
FROM information_schema.columns
WHERE table_name = 'coba'
AND column_name NOT IN('pk')
), ',') || ' FROM binsar.coba where pk=''1'';';


Output:
SELECT '300',nourut,nourutku,unit_id FROM binsar.coba where pk='1';


So, new rows can be added with ease. We do not need to write all the field names,
just need to write down the field we do not want, also without hard coding the SELECT script.

In a stored function (PL/pgSQL), I use EXECUTE 'statement' for all tables in a database.

CREATE OR REPLACE FUNCTION cobadeh(pk character, newpk character,tabname character)
RETURNS character AS
$BODY$
DECLARE
t1 character varying (300);
t2 character varying (300);
BEGIN
t1='INSERT INTO binsar.' || tabname;
SELECT ' SELECT ' || '''' || newpk|| '''' || ',' || array_to_string(ARRAY(SELECT '' || column_name
FROM information_schema.columns
WHERE table_name = 'coba'
AND column_name NOT IN('pk')
), ',') || ' FROM binsar.coba WHERE pk=' || '''' ||pk ||'''' INTO t2;
EXECUTE t1 || t2;
RETURN t1 || t2;
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE
COST 100;
ALTER FUNCTION cobadeh(character,character,character) OWNER TO binsar;

then tests it:

select * from binsar.cobadeh('1','300', 'coba');
select * from binsar.cobadeh('3','ABC', 'coba');

will run these command:
"INSERT INTO binsar.coba SELECT '300',nourut,nourutku,unit_id FROM binsar.coba WHERE pk='1'"
INSERT INTO binsar.coba SELECT 'ABC',nourut,nourutku,unit_id FROM binsar.coba WHERE pk='3'


The table now has new rows:
select * from binsar.coba ;
"1 ";1;1;"1 "
"2 ";2;2;"2 "
"3 ";1;1;"1 "
"300 ";1;1;"1 "
"ABC ";1;1;"1 "

that's.

Minggu, 09 Januari 2011

Net-SNMP on FreeBSD

1. Installation using ports
# cd /usr/ports/net-mgmt/net-snmp
# make install clean

2. List directories and files name that contain "snmpd"
# pkg_info -L net-snmp-5.4.2.1 | grep snmpd
...
/usr/local/sbin/snmpd
/usr/local/share/snmp/snmpd.conf.example
/usr/local/etc/rc.d/snmpd

3. Create snmpd.conf file from the example
# mkdir /usr/local/etc/snmp
# cp /usr/local/share/snmp/snmpd.conf.example /usr/local/etc/snmp/snmpd.conf

4. Edit the file
# vi /usr/local/etc/snmp/snmpd.conf

and modify from:
# sec.name source community
com2sec local localhost COMMUNITY
com2sec mynetwork NETWORK/24 COMMUNITY

become:
# sec.name source community
com2sec local localhost private
com2sec mynetwork 192.168.123.0/28 public

5. Modify rc configuration file
#vi /etc/rc.conf
then add this
snmpd_enable="YES"

6. Start snmp daemon
Check if port 161 active
#sockstat -4 | grep 161
---nothing displayed

Ready to start snmpd
# /usr/local/etc/rc.d/snmpd start
Starting snmpd.

# sockstat -4 | grep 161
root snmpd 16217 10 udp4 *:161 *:*

# ls /var/run | grep snmpd
snmpd.pid

Read the log file
# cat /var/log/snmpd.log

7. Is it works?
# snmpwalk -c private -v1 localhost | grep Address
# snmpwalk -c private -v2c localhost HOST-RESOURCES-MIB::hrSWRunName
# snmpwalk -c public -v1 192.168.123.1 sysDescr
SNMPv2-MIB::sysDescr.0 = STRING: OpenBSD host1.mydomain.com 4.7 GENERIC.MP#449 i386

# snmpwalk -c public -v1 192.168.123.4 sysDescr
SNMPv2-MIB::sysDescr.0 = STRING: FreeBSD host2.mydomain.com 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Thu Jan 15 16:15:25 WIT 2009 root@host2.mydomain.com:/usr/src/sys/i386/compile/BINSARBSD i386

# snmpwalk -c public -v1 192.168.123.1 hrSystemUptime.0
HOST-RESOURCES-MIB::hrSystemUptime.0 = Timeticks: (15403177) 1 day, 18:47:11.77

# snmpget -c public -v1 192.168.123.4 hrSystemUptime.0
HOST-RESOURCES-MIB::hrSystemUptime.0 = Timeticks: (1598771) 4:26:27.71

# snmpget -c public -v2c 222.xxx.yyy.zzz sysUpTime.0
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (15976711) 24 day, 20:22:47.11

# snmpget -c private -v1 localhost .1.3.6.1.4.1.2021.100.6.0
# snmpwalk -c public -v1 192.168.123.1 .1.3.6.1.4.1.2021.100.6.0
# snmpstatus -c private -v2c localhost
[UDP: [0.0.0.0]->[127.0.0.1]:161]=>[FreeBSD host2.mydomain.com 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Thu Jan 15 16:15:25 WIT 2009 root@host2.mydomain.com:/usr/src/sys/i386/compile/BINSARBSD i386] Up: 0:44:48.24
Interfaces: 3, Recv/Trans packets: 116458/74594 IP: 85052/74569
1 interface is down!

# snmpstatus -c public -v2c 192.168.123.1
[UDP: [0.0.0.0]->[192.168.123.1]:161]=>[OpenBSD host1.mydomain.com 4.7 GENERIC.MP#449 i386] Up: 1 day, 19:15:49.53
Interfaces: 7, Recv/Trans packets: 337656/162218 IP: 219127/159655
3 interfaces are down!

# snmptest -c public -v 1 localhost
Variable: system.sysDescr.0
Variable:
Received Get Response from UDP: [0.0.0.0]->[127.0.0.1]:161
requestid 0xCC474EF errstat 0x0 errindex 0x0
SNMPv2-MIB::sysDescr.0 = STRING: FreeBSD host2.mydomain.com 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Thu Jan 15 16:15:25 WIT 2009 root@host2.mydomain.com:/usr/src/sys/i386/compile/BINSARBSD i386
Variable:

You may be interested in these OID:
# snmpget -c private -v1 localhost .1.3.6.1.2.1.1.1.0

probably these too
1.3.6.1.2.1.2.2.1.10.1
1.3.6.1.2.1.2.2.1.16.1
1.3.6.1.4.1.2021
1.3.6.1.4.1.2021.2
1.3.6.1.4.1.2021.100.6.0
1.3.6.1.4.1.2021.100.11.0

In this writing, snmpd works with SNMP version 1 and 2c and some snmpd command has been used.