Thursday, September 19, 2013

Compassion between Traditional vs Big Data Approach


  1. here listed out important characteristics difference between traditional RDBMS and Big Data 

    Characteristics of Traditional vs Big Data approaches


    Characteristics
    Data Management
    Relational Data
    Big Data


    Architecture
    Centralized
    Distributed
    Data Volume
    Tera-bytes
    Peta-bytes to Exa-bytes
    Data Relational
    Known relation
    Unknown/complex relation
    Data Model
    Static or schema-based
    Dynamic or Schema-lessa
    Data veracity
    Related Data
    Messy/imprecise
    Data Velocity
    Generated by human interaction
    Machine generated data sensor,medical and FSI
    Data source
    Known
    Heterogeneous
    Scalability
    Nonlinear
    Liner
    Data processing
    Interactive
    Batch and stream processing











Wednesday, September 11, 2013

Type of Big Data Analytics


Big Data analytics are emerging to get the actionable data from huge and heterogeneous data. Different  Big Data analytics used in these digitized world.  mainly classified into three categories 
  1. Business analytics
  2. operational analytics
  3. social media analytics
different analytics used in each categories to Extracting business or mission intelligence data from massive amount of data.  

  • Business analytics
    • Predictive Analysis 
    • Behavioral Analysis
    • Comparative Analysis
    • Marketing Analysis
  • operational analytics
    • Fraud Analysis
    • Risk Analysis
    • Customer churn Analysis
    • Retail and sales Analysis
  • social media analytics
    • sentimental analysis


      For  example in Health-care business different analysis used i.e 

          Descriptive Analysis
          Predictive Analysis
          Prescriptive Analysis










Thursday, September 5, 2013

Postgresql - Triggers on DDL Statements

PostgreSQL 9.3

Awaiting DDL Triggers i.e EVENT TRIGGERS  features  finally  come live with PostgreSQL 9.3

Yes!!.. now we can have a Trigger on DDL Statements like Oracle does.

Syntax                                                                                                                               

CREATE - EVENT TRIGGER 
CREATE EVENT TRIGGER trigger_name
  ON event
  [ WHEN condition IN (values)  ]
  EXECUTE PROCEDURE function_name()

event : ddl_command_startddl_command_end and sql_drop 

ddl_command_start
  •  Trigger function ( i.e function_name() ) would be called before execution of CREATE/ALTER/DROP command.
ddl_command_end
  • Trigger function ( i.e function_name() ) would be called after execution  of CREATE/ALTER/DROP command.
sql_drop 
  • Trigger function ( i.e function_name() ) would be called Just before "ddl_command_end" event triggered.
  • Only for the DROP commands
  • To view the list of objects deleted by pg_event_trigger_dropped_objects()

condition : tg_tag. ( It will return calling trigger command ) 
values : CREATE TABLE, CREATE FUNCTION complete list of values 
function_name : A normal pl/pgsql function with return type as event_trigger & it not needed to return any value.

Ex:
CREATE TRIGGER FUNCTION

CREATE OR REPLACE FUNCTION fn_ddl_trigger()
  RETURNS event_trigger
 LANGUAGE plpgsql
  AS $$
BEGIN
  RAISE EXCEPTION 'Current running DDL command is : % ', tg_tag;
END;
$$;

CREATE EVENT TRIGGER 1)  Simple  

CREATE EVENT TRIGGER ddl_trigger ON ddl_command_start
   EXECUTE PROCEDURE fn_ddl_trigger();

OUTPUT 1)

root@boss[~]#su postgres
postgres@boss:$psql
postgres=# create table table1 (empname varchar(200));
NOTICE:  Current running DDL command is: CREATE TABLE
CREATE TABLE


CREATE EVENT TRIGGER 2) With 'WHEN' condition
      
CREATE EVENT TRIGGER ddl_trigger ON ddl_command_start WHEN tg_tag in (ALTER_TABLE, DROP_TABLE)
EXECUTE PROCEDURE fn_ddl_trigger();


OUTPUT 2)  trigger function will get called only ALTER and DROP, not for CREATE TABLE, so no messgae to display.


root@boss[~]#su postgres
postgres@boss:$psql
postgres=# create table table2 (empname varchar(200));

CREATE TABLE
ALTER - EVENT TRIGGER

ALTER EVENT TRIGGER name DISABLE
ALTER EVENT TRIGGER name ENABLE [ REPLICA | ALWAYS ]
ALTER EVENT TRIGGER name OWNER TO new_owner
ALTER EVENT TRIGGER name RENAME TO new_name

DROP - EVENT TRIGGER

DROP EVENT TRIGGER [ IF EXISTS ] name





Tuesday, September 3, 2013

MySQL start FAILED on VM Virtual Box

Recently  I had on issue with MySQL Start-up with VM. my environment was
Host OS   : Debian / BOSS
Guest OS : CentOS
Application running in CentsOS. We tried to replicate the same in Debian/BOSS but wouldn't because some missing packages. so decided to go with Virtual Machine concept.

When we boot the CentOS from Debian/BOSS, MySQL Failed to start.

As we followed my error.log we managed to start the MySQL.

1) Network Adapter settings in your VM settings to be bridged mode

2) Change bind address  in /etc/mysql/my.cnf to 0.0.0.0
bind-address = 0.0.0.0
 start the server by /etc/init.d/mysql start

3) gone through the log file (img),


MySQL error log  running on VM

got few error which was related to permission denied.


  • Error 1)
/usr/libexec/mysqld: File '/var/lib/mysql/log-bin.index' 
not found (Errcode: 13)
130930 15:27:22 [ERROR] Aborting
Solution
root@boss[~]# chmod -R 777   /var/lib/mysql/log-bin.index 
  • Error 2) 


innoDB : Operating Systems error number 13 in a file operation. the Error means mysqld doesn't have the access rights to the directory
File name ./ibdata
File operation call : 'open'
Cannot continue operation

Solution
root@boss[~]# chomd -R 777 `locate ibdata`


  • Error 3)
[ERROR] /usr/libexec/mysqld: Can't create/write to file '/var/run/mysqld/mysqld.pid' (Errcode: 13)
[ERROR] Can't start server : can't create PID file: Persmission denied

Solution
root@boss[~]#  chmod -R 777 /var/run/mysqld/mysqld.pid

root@boss[~]#  /etc/init.d/mysql start 

now MySQL started successfully