Category Archives: Linux

df: `/root/.gvfs’: Permission denied

When you run the df command as a non root user, you can sometimes get the following error:

df: `/root/.gvfs': Permission denied

When you ran the df command you can pass the file system types which you would like to exclude from it, in this case it is ‘fuse.gvfs-fuse-daemon’ just as follows:

df -x fuse.gvfs-fuse-daemon

Email alert for low disk space v2

A while back I wrote a post about a simple low disk space script that will email you if a specific partition on your system has free space lower than a percentage you specified. Now I improved that script, adding some more features:

– excluding some partitions (be careful here not too exclude to much)

– sending email to many recipients

– check if emails were already sent, then it would spam the users with emails

The script is below:

#!/bin/sh 
EXCLUDE_PARTITIONS="partition1\|partition2" 
WARNING=85 
RECIPIENTS="user1@domain.com user2@domain.com" 
DF=( `df -h | grep % | tail -n +2 | awk '{print $(NF-2)" "$(NF-1)" "$(NF)}' | sed 's/%//g' | sed 's#/\(.*\)#/\1 slash-\1#g' ` ) 
for ((i=1;i<${#DF[@]};i=i+4)) 
do 
  if [[ ${DF[i]} -gt $WARNING ]] 
  then 
    if [ ! -f /tmp/disk-warning-for-${DF[i+2]} ] 
    then 
      touch /tmp/disk-warning-for-${DF[i+2]} 
      echo "" | mail -s "SH Alert: Critical: `hostname` - ${DF[i+1]} has only ${DF[i]}% (${DF[i-1]}) free space" $RECIPIENTS 
    fi 
  else 
    if [ -f /tmp/disk-warning-for-${DF[i+2]} ] 
    then 
      echo "" | mail -s "SH Alert: Clear: `hostname` - ${DF[i+1]} has only ${DF[i]}% (${DF[i-1]}) free space" $RECIPIENTS 
      rm /tmp/disk-warning-for-${DF[i+2]} 
    fi 
  fi 
done

As in the previous post, you need to make the script executable:

chmod +x /path/to/script

And add it to the cronjob using “cronjob -e”:

# Minute   Hour   Day of Month       Month          Day of Week        Command
# (0-59)  (0-23)     (1-31)    (1-12 or Jan-Dec)  (0-6 or Sun-Sat)
*/10 * * * * /path/to/script

Note: This was tested on Oracle Linux 5/6 and is working great. For other distro’s you might need to change the email syntax, the order of the df columns, etc.

Fast method for finding differences between two unsorted files

At work I had two files: one with 13 000 records (filea) and one with 300 000 records (fileb). Every record was on a new line in the file. I was looking for a method of finding which records from filea are not in fileb. Because of the huge number of records per file, the “grep -v -f filea fileb” was taking forever and also I would need the files to be sorted before executing the grep.

After a few searches on the world wide web, I found a very quick way of doing this with awk:

1
awk 'NR == FNR { A[$0]=1; next } !A[$0]' fileb filea

If you want to find out more about this fast and easy solution (no need to presort the files) I found a similar example with an explanation here.

GNU utilities native in windows

For all you Linux fans out there, that use a lot of linux commands (sed,ls,awk,etc) and would like to have this also on linux, you can give UnxUtils a try. The big differences between UnxUtils and Cygwin is that UnxUtils uses executables that are only dependent to Microsoft C-runtime (msvcrt.dll).

All you have to do is:

1. Download the zip file from owners site

2. Unzip the file in any location (preferably C:\Program Files (x86)\UnxUtils )

3. Add the path C:\Program Files (x86)\UnxUtils\usr\local\wbin to the System Variables PATH (Start – Control Panel – System – Advanced system settings – Advanced – Environment Variables – System variables – PATH – edit)

You can find a complete list of programs that are ported on the UnxUtils site.

Delete old S_NQ_ACCT/S_NQ_DB_ACCT partitions

In the previous post I’ve described how to partition the S_NQ_ACCT and S_NQ_DB_ACCT tables used by OBIEE for Usage Tracking. Now I’m gonna describe how to delete partitions that are X days old.

When we partitioned the table, we configured it to be done daily. Now if we are going to issue the following command:

conn DEV_BIPLATFORM;
SELECT * FROM user_tab_partitions;

we are gonna notice that Oracle Database created the HIGH_VALUE as “TO_DATE(‘ 2013-11-09 00:00:00’, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)”. If you want to delete all partitions that are 32 days old, I’ve used the following sql script:

SET serveroutput ON
SET feedback off
variable v_days_to_keep NUMBER;
exec :v_days_to_keep := 32;
 
DECLARE
    v_statement VARCHAR2(4000);
BEGIN
    FOR x IN (SELECT TABLE_NAME,PARTITION_NAME,HIGH_VALUE FROM user_tab_partitions WHERE TABLE_NAME = 'S_NQ_DB_ACCT') LOOP
      IF x.PARTITION_NAME != 'PART1'
        THEN
          v_statement := '
BEGIN
    IF SYSDATE - ' || x.HIGH_VALUE || ' > '|| :v_days_to_keep || '
      THEN
        EXECUTE IMMEDIATE ''ALTER TABLE "' || x.TABLE_NAME || '" DROP PARTITION "' || x.PARTITION_NAME || '" UPDATE GLOBAL INDEXES'';
    END IF;
END;';
          EXECUTE IMMEDIATE v_statement;
      END IF;
    END LOOP;
    EXECUTE IMMEDIATE 'alter table S_NQ_DB_ACCT disable CONSTRAINT "FK_S_NQ_DB_ACCT"';
    FOR x IN (SELECT TABLE_NAME,PARTITION_NAME,HIGH_VALUE FROM user_tab_partitions WHERE TABLE_NAME = 'S_NQ_ACCT') LOOP
      IF x.PARTITION_NAME != 'PART1'
        THEN
          v_statement := '
BEGIN
    IF SYSDATE - ' || x.HIGH_VALUE || ' > '|| :v_days_to_keep || '
      THEN
        EXECUTE IMMEDIATE ''ALTER TABLE "' || x.TABLE_NAME || '" DROP PARTITION "' || x.PARTITION_NAME || '" UPDATE GLOBAL INDEXES'';
    END IF;
END;';
          EXECUTE IMMEDIATE v_statement;
      END IF;
    END LOOP;
    EXECUTE IMMEDIATE 'alter table S_NQ_DB_ACCT enable novalidate CONSTRAINT "FK_S_NQ_DB_ACCT"';
END;
/

I’ve tested this and it’s working perfectly on OBIEE 11.1.1.7.1 with DB 11.2.0.3. All you have to do now is set a job in the DB or sh script in the OS to run daily. I’ve went with the second option, because it’s simpler for me to receive email alerts if anything goes wrong with the script. I’ve also added some output to the script so I know when something is deleted. The sh script is below. Please change the password and tns_entry accordingly to you environment.

export ORACLE_HOME=/u01/app/oracle/product/11.2.0/dbhome_1
export PATH=$ORACLE_HOME/bin/:${PATH}
export TNS_ADMIN=$ORACLE_HOME/network/admin
 
before=`sqlplus -s DEV_BIPLATFORM/pass@tnsentry << EOFSQL
set heading off
set pages 0
select count(1) from user_tab_partitions;
exit;
EOFSQL`
 
sqlplus -s DEV_BIPLATFORM/pass@tnsentry << EOFSQL
Set serveroutput on
set feedback off
variable v_days_to_keep number;
exec :v_days_to_keep := 32;
 
declare
    v_statement varchar2(600);
begin
    for x in (select TABLE_NAME,PARTITION_NAME,HIGH_VALUE from user_tab_partitions where TABLE_NAME = 'S_NQ_DB_ACCT') loop
      if x.PARTITION_NAME != 'PART1'
        then
          v_statement := '
BEGIN
    IF SYSDATE - ' || x.HIGH_VALUE || ' > '|| :v_days_to_keep || '
      THEN
        EXECUTE IMMEDIATE ''ALTER TABLE "' || x.TABLE_NAME || '" DROP PARTITION "' || x.PARTITION_NAME || '" UPDATE GLOBAL INDEXES'';
    END IF;
END;';
          execute immediate v_statement;
      end if;
    end loop;
    EXECUTE IMMEDIATE 'alter table S_NQ_DB_ACCT disable CONSTRAINT "FK_S_NQ_DB_ACCT"';
    for x in (select TABLE_NAME,PARTITION_NAME,HIGH_VALUE from user_tab_partitions where TABLE_NAME = 'S_NQ_ACCT') loop
      if x.PARTITION_NAME != 'PART1'
        then
          v_statement := '
BEGIN
    IF SYSDATE - ' || x.HIGH_VALUE || ' > '|| :v_days_to_keep || '
      THEN
        EXECUTE IMMEDIATE ''ALTER TABLE "' || x.TABLE_NAME || '" DROP PARTITION "' || x.PARTITION_NAME || '" UPDATE GLOBAL INDEXES'';
    END IF;
END;';
          execute immediate v_statement;
      end if;
    end loop;
    EXECUTE IMMEDIATE 'alter table S_NQ_DB_ACCT enable novalidate CONSTRAINT "FK_S_NQ_DB_ACCT"';
end;
/
exit;
EOFSQL
 
after=`sqlplus -s DEV_BIPLATFORM/pass@tns_entry << EOFSQL
set heading off
set pages 0
select count(1) from user_tab_partitions;
exit;
EOFSQL`
 
deleted=`expr $before - $after`
[[ ! $deleted -eq 0 ]] && echo "$deleted paritions deleted."

Segmentation fault (11) with mod_weblogic and ssl weblogic

Recently I’ve stumbled upon a strange one. We have setup an OHS in front of OBIEE, on different machine to enable caching, SSL and fast failover (switching between two synced OBIEE servers). We use /em and /console links to redirect to the weblogic Enterprise Manager and Administration Console. Until now we used the standard non SSL ports for this. We got a notification that all traffic from our server should be encrypted using SSL. As OHS and OBIEE are on different servers, we needed to configure SSL on the weblogic ports 7001 and 9704.

Enabling SSL on Weblogic was easy, as I will explain in a later post. Now I needed to configure mod_weblogic to communicate to weblogic on the ssl port. As our OHS (Oracle HTTP Server) 11.1.1.6.0 installation is standard, I tried to enable the mod_weblogic to connect to weblogic on a ssl port, using the following configuration:

<Location /console>
  <IfModule mod_weblogic.c>
    SetHandler weblogic-handler
    WebLogicHost hostname
    WebLogicPort 7002
    SecureProxy ON
    WlSSLWallet "path_to_oracle_wallet"
  </IfModule>
</Location>

I kept getting the following error every time a request was made:

[Tue Dec 03 13:54:47 2013] [notice] child pid 21617 exit signal Segmentation fault (11)

This was very strange because everything was working fine, I was able to login to the console. After a little digging it seems that the mod_wl_ohs.so plugin is to blame. After I downloaded the plugin from the Oracle Technology Network and modified the mod_wl_ohs.conf to include the new plugin, the error in log file was gone, and it seemed that everything was working faster.

Will have to see if this interferes with normal OBIEE operation…

Configure RestFull Services on Apex Listener

I had to configure Apex Listener to work with RestFull Services. After a lot of hours of debugging I managed to make my Apex Listener on Weblogic to respond to the services.

A think that helped me a lot in all this debugging was the Apex Standalone server (if your environement is not in production or dev you can modify that as well) started in the debug mode. Basically you have to change the defaults.xml file in the Apex Listnere configdir and enable the on screen display:

<entry key="debug.printDebugToScreen">true</entry>

The steps to make it all work are the following:

1. Change directory to your apex installation and connect to the database and run apex_rest_config.sql and set the password for the APEX_LISTENER and APEX_REST_PUBLIC_USER users.
Note: If you do this step after step 2, be sure to unlock the two users because they will get locked when Apex Listener will try to connect with the wrong password.

2. Deploy Apex to Weblogic according to the official documentation here: http://docs.oracle.com/cd/E37099_01/doc/doc.20/e25066/install.htm#autoId17
Note: Be sure to enter the correct users and passwords.

3. In our case, there were a lot of Workspaces created before installing and enabling RestFull Services. I think that for all the Workspaces that are created before enabling RestFull Services or created on the same Apex Instance but from another deployment that does not have RestFull Services (like Apache) the url-mappings are not correct. If you try to test the service on a old Workspace you will get the following error:

Request Path passes syntax validation
Mapping request to database pool: PoolMap [_poolName=apex, _regex=null, _workspaceIdentifier=null, _failed=false, _lastUpdate=-1, _template=null, _type=REGEX]
Applied database connection info
Attempting to process with PL/SQL Gateway
Not processed as PL/SQL Gateway request
Attempting to process as a RESTful Service
Determining if request can be dispatched as a Tenanted RESTful Service
Request path has one path segment, continuing processing
No Tenant Principal established yet, continuing processing
APEX_LISTENER pool exists, continuing processing
No matching tenant found for: travel, cannot dispatch
No candidate found for: GET travel/hr/empinfo/ in context: http://localhost:8080/apex/

In the above example travel is the Workspace.

For the restfull services to work on old workspaces you will have to create the mappings manually according to the documetnation: http://docs.oracle.com/cd/E37099_01/doc/doc.20/e25066/config.htm#autoId3. In our example you should run:

java -jar apex.war map-url --type base-path --workspace-id travel /travel apex

And that should fix the issue.

Adding new listeners for Oracle Database

I got a request today, to make available the same Oracle Database instance on an another port besides the default one. One simple but not so elegant way to make this happen (if you have root access) is to redirect the traffic from one port to the one you already have setup using this command:

iptables -t nat -I PREROUTING --src 0/0 --dst 10.0.1.15 -p tcp --dport 1522 -j REDIRECT --to-ports 1521

The other way, the more elegant way, is to create another listener on the requested port and tell the database to register itself to that listener. Here are the steps you need to follow, after you have run “. oraenv” to setup your environment:

1. Create the new listener on the desired port:

srvctl add listener -l LISTENER2 -p TCP:1522 -o /u01/app/oracle/product/11.2.0.3/dbhome_1

-l = the listener name, LISTENER is most probably already taken
-p = the protocol and port to use
-o = Oracle Home

2. Start the listener:

srvctl start listener LISTENER2

3. Now you need to modify the local_listener parameter in the database. If you need to set multiple ports, then you will need to use the tnsname for that. If you try to input the string directly in the database you will receive an ORA-00972: identifier is too long. So add the following entry in the $ORACLE_HOME/network/admin/tnsnames.ora file:

ORCL_1521_1522 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.1.15)(PORT = 1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = 10.0.1.15)(PORT = 1522))
    )
  )

4. Configure the local_listener in the database:

[oracle@srv ~]$ sqlplus / as sysdba
SQL> ALTER SYSTEM SET local_listener = 'ORCL_1521_1522' SCOPE=MEMORY SID='ORCL11';

System altered.

5. Register the entries with the listener:

SQL> alter system register;

Now you should see when running ‘lsnrctl status LISTENER2’ you should see the new registered database in the listener.

As far as I remember the ‘lsnrct start’ command will only start the default listener, so don’t forget to start the second listener as well. If you have configuret the automatic db start on boot, then you will have to modify the startup scripts (/etc/init.d/dbora file fi you followed the Official Documentation) and add at startup and shutdown the appropriate commands.

< Line 28, Column 49 >: XML-20190: (Fatal Error) Whitespace required.

While configuring Active Directory in weblogic and obtaining group membership from Database tables, after numerous configurations, deletes and restarts, when I tried configuring the virtual adapter using libovdadapterconfig.sh I ran in the following error:

Exception in thread "main" org.xml.sax.SAXParseException: <Line 28, Column 49>: XML-20190: (Fatal Error) Whitespace required.

After a lot of digging I’ve noticed that the adapterTemplate file did not have a space between the xml variables. After adding the space everything went fine.

Hope this helps you not to loose too much time on a simple mistake.