pajama

PROBLEM

When using Kernel Asynchronous I/O (KAIO) with IBM Informix Dynamic Servers, you might experience the following message in the database server message log:

KAIO out of resource errno=11
or
KAIO out of resource errno=35

Note: Check Related information for more detail. 

• All configurable kernel parameters must be specified using an integer value or a formula consisting of a valid integer ｅxpression.

• The maximum and/or default values of certain parameters may change between releases or to vary between 32-bit and 64-bit processors.

• Before altering the value of any configurable kernel parameter, be sure you know the implications of making the change.
• Never set any system parameter to a value outside the allowable range for that parameter (SAM refuses to store values outside of the allowable range).
• Many parameters interact, and their values must be selected in a balanced way.

PROBLEM

Database performance is poor because of input/output (I/O) requests. The IBM® Informix® Dynamic Server™ message log might display one of these error messages:

KAIO out of OS resources, errno = 11

or   

KAIO out of OS resources, errno = 35

Limits of the  IFMX_HPKAIO_NUM_REQ  environment variable.
Minimum Value	10
Maximum Value	5000
Default Value	1000

Note: These values may change in future versions of the operating system or database server.

Example:

This example shows how to increase the number of concurrent KAIO requests to 2300 from the default of 1000 using Korn shell:

$ IFMX_HPKAIO_NUM_REQ=2300 
$ export IFMX_HPKAIO_NUM_REQ

• output of oninit -v:
  
  Checking group membership to determine server run mode...succeeded
  Reading configuration file '/opt/informix/etc/onconfig.test'...succeeded
  Creating /INFORMIXTMP/.infxdirs...succeeded
  Checking config parameters...succeeded
  Allocating and attaching to shared memory...FAILED
  
• messages in online.log:
  
  21:45:30 IBM Informix Dynamic Server Started.
  21:45:30 size of resident + virtual segments 10443875920 + 4831838208 > 9663676416
  total allowed by configuration parameter SHMTOTAL

The size of the resident and virtual portions of shared memory exceed the shared memory limit set by the onCONFIG parameter SHMTOTAL.

Resident portion of shared memory 
The resident portion of the database server shared memory stores the following data structures that do not change in size while the database server is running:

• Shared-memory header
• Buffer pool
• Logical-log buffer
• Physical-log buffer
• Lock table
  

• Internal tables
• Big buffers
• Session data
• Thread data (stacks and heaps)
• Data-distribution cache
• Dictionary cache
• SPL routine cache
• SQL statement cache
• Sorting pool
• Global pool
  

Estimating the Size of the Resident Portion of Shared Memory 
Estimating the Size of the Virtual Portion of Shared Memory

• Increase the value of the onCONFIG parameter SHMTOTAL if possible.
• Decrease the value of one or more of these onCONFIG parameters:
  
  - BUFFERPOOL ( the buffers value )
  - LOCKS
  - SHMVIRTSIZE

Once the ping timeout is written to the online.log file of the Primary/Secondary instance (see the sample output below), user sessions return to normal work.

11:27:57  DR: ping timeout                                             
11:27:57  DR: Receive error                                            
11:27:57  ASF Echo-Thread Server: asfcode = -25582: oserr = 4: errstr =
: Network connection is broken.                                        
11:27:57  DR_ERR set to -1                                             
11:27:59  DR: Turned off on primary server    

When data replication is established, primary and secondary regularly exchange ping messages. If the ping acknowledge is not received by the time when DRTIMEOUT is elapsed, a server re-sends ping message three more times and then reports ping timeout and turns off the DR subsytem. From this, the time span between first ping and the "DR: ping timeout" message can be as large as (DRTIMEOUT x 4).

For example, if DRTIMEOUT is set to be 180 second, it will take 12 minutes before DR is turned off.

Scenario #1:
Although with asynchronous replication transactions do not wait for acknowledgement from HDR secondary after the logical log record was put in DR buffer, when there is a transmission failure, the DR buffer may fill up pretty quickly (the time required for that depends on DRTIMEOUT value, LOGBUFF value and the activity that the instance is having). Until DR is not turned off, a user session has to wait until DR buffer has enough space for the logical log record.

Scenario #2:
In addition to the above scenario, a checkpoint can be requested on Primary between the first ping failure and the time when the "DR: ping timeout" message is reported. The checkpoints are synchronous between Primary and Secondary regardless of the DRINTERVAL value. once checkpoint is requested, it will prevent any threads from entering the critical section. The instance will remain blocked until checkpoint acknowledgment is received from the Secondary or until DR is turned off.

For scenario #1 check if the corresponding user thread demonstrates a stack similar to the following:

Stack for thread: 73 sqlexec                  
base: 0x0700000011abc000                      
len: 69632                                    
pc: 0x00000001000370f4                        
tos: 0x0700000011acafe0                       
state: sleeping                               
vp: 8                                         
                                              
0x00000001000370f4 (oninit)yield_processor_mvp
0x0000000100041f30 (oninit)mt_yield           
0x000000010076a5ac (oninit)cdrTimerWait
0x0000000100716908 (oninit)dr_buf_deq_int
0x00000001001fe3c0 (oninit)dr_logcopy         
0x00000001001f2d0c (oninit)logwrite
0x000000010011b7c4 (oninit)log_put            
0x0000000100121384 (oninit)logm_write         
0x00000001001f3e68 (oninit)logputx            
0x000000010017137c (oninit)rscommit           
0x000000010022b70c (oninit)iscommit           
0x00000001002865e4 (oninit)sqiscommit         
0x0000000100533d38 (oninit)committx           
0x0000000100536480 (oninit)commitcmd          
0x000000010053b01c (oninit)excommand          
0x000000010042893c (oninit)sq_execute         
0x000000010026becc (oninit)sqmain             
0x00000001002d51a4 (oninit)listen_verify      
0x00000001002d33b8 (oninit)spawn_thread       
0x0000000100e0b59c (oninit)startup       

For scenario #2 check the 'onstat -g ath' output and see if the user threads are having "cond wait cp" status.

To resolve the problem it may be required to:

1) Fix any problems that can cause data transmission issues between Primary and HDR Secondary (e.g. increase network reliability and throughput)

2) Decrease the value of DRTIMEOUT configuration parameter.

Note: increasing the LOGBUFF may also help to reduce the blockage time, however having a large logical log buffer may result in data loss in case of the Primary failure.

ping timeout,received error,send error

ping timeout will occur if "DR_MSG_PING" can't flow between primary and secondary,or ack duration exceed 4*DRTIMEOUT."ping timeout" is a message type in DR BUFFER QUEUE,so it require waiting for dr buffer space,therefor PING TIMEOUT maybe occur due to dr buffer is full or its priority is too lower than logical log buffer.

Logical log buffer can't be transfer maybe lead to the 'ping timeout'.

HDR environment

DR BUFFER size is same as logical log buffer,and "DR_MSG_PING" save in dr buffer,so we can configure LOGBUFF to adjust DR BUFFER.

Primary server send logical log to Secondary server to keep consistent data as following description.
1.primary : logical log buffer -> dr buffer
2.primary : dr_prsend thread send these logical log to dr buffer on secondary server across network using TCP/IP.
3.secondary:dr_secrecv thread received those logical log in secondary.

HDR primary server and secondary server will ping each other and must waiting for a acknowledgment during a appointed times ,otherwise HDR relation will be broken due to "ping timeout" error .The ack duration is 4 times as DRTIMEOUT value.

To avoid "ping timeout" occur according to following mention.

1.Increasing LOGBUFF value to adjust dr buffer size and lay more signal message.
2.Secondary server hang maybe lead to the "ping timeout" due to dr buffer can't be received immediately or DR_MSG_PING lower priority.
3.Long checkpoint duration in secondary server.

Error message: "FAILED: Fetch statement failed: Encoding or code set not supported. "

error -79783

JDBC version 3.50.JC1 introduced the following APAR:

IC49877 - 
JDBC DRIVER ALLOWS INSERTION OF INVALID CHARACTERS FOR CHARACTER SET. 
http://www-01.ibm.com/support/docview.wss?uid=swg1IC49877 

While the behavior for this APAR is correct, it might lead to problems for those customers who use Native Multibyte data on the application side but store them into en_us locale on the server side. 

In some case, for customer who are using zh_tw.big5 locale on the server side; but stored illegal characters in the table will also be effected.

Error message happened right after upgrade.

Since version JDBC 3.50.JC5, user can set a flag IFX_USE_STRENC to switch to old style of encoding.

Here's an example on how to use it: 
"jdbc:informix-sqli://inst:port:dbname:informixserver=XXX;user=informix;password=XX;DB_LOCALE=en_us.819;IFX_USE_STRENC=true;"

INTRODUCTION

Most customers using Informix products are used to having an option which gives them the version of the product being used. This is usually done at the command line using the "-V" option or using a special utility.

For example: esql -V would give the version of esql or the utility ifx_getversion can be used to display the version of the ClientSDK.

It is also possible to obtain the version of the JDBC driver contained in the ifxjdbc.jar file from the command line.


BEFORE YOU BEGIN

Make sure you have set the CLASSPATH variable correctly to include the location of ifxjdbc.jar ( or ifxjdbc-g.jar ) and that your PATH has the JDK in it.


STEPS

Execute the following

java com.informix.jdbc.Version

% java com.informix.jdbc.Version 
IBM Informix JDBC Driver Version 2.21.JC5

PROBLEM

You receive one or more -408 errors in your Informix message log. The generic error description is -invalid message type received from the sqlexec process.The informix utility finderr states that it is some form of a communication error but you are still unsure what the problem really is. Further, there is no associated ISAM code in the message log - or the ISAM code reported is 0. 


POSSIBLE CAUSES

This is a generic message which can have several causes -

• An older version of CSDK or some other tool is trying to access the database - the server receives in the communication channel from the client, information that it does not know what to do with.
• A simple TELNET to the Informix port. Make sure that the telnet software is compatible with the database engine in use and is using the correct port
• If an 4gl application was compiled on the development machine under a different installed version of 4gl than is present on the runtime machine NOTE: This usually is the singular most common cause of this error.
• If the Informix products are installed either incorrectly, or in the wrong order. An incorrect install may be due to installing the product as a user other than 'root'. This may also be due to attempting to install several products at one time by running tar or cpio off all the media at one time, and then finishing the install by running the various install scripts. The correct way to install would be to transfer the tar or cpio file from the first set of media, then run the install script for that product, and then transfer from the media for the second product followed by its install script, and so on.

• If the sqlexec process that was launched against the database engine by the ESQL application has been killed. (Look for 'Process Aborted Abnormally')error messages in the online log to see if this is a potential problem.) Typically this will only happen via a unix 'kill' command.

• Data or index corruption. Recreating all involved tables, indexes and views. Possibly running an oncheck -cID against the database may help.

• If the product is installed on an OS that is different than that for which it was ported. For instance, ESQL/C ported for Solaris 2.3 will NOT run reliably on Solaris 2.4 or Solaris 2.5, though it may appear to work on some occasions. This is because the OS versions are not binary compatible.

• In association with memory allocation errors (-208) because cursors and/or prepared statements are getting reopened/reprepared multiple times in a loop without the original cursor or statement ID being freed.

To obtain a list of the tables in a particular chunk, below sql can be used:
select dbinfo( "DBSPACE" , pe_partnum ) dbspace,

dbsname[1,12],

tabname,

pe_chunk,

pe_offset start,

pe_size size

from sysptnext, outer systabnames

where pe_partnum = partnum

and pe_chunk = <chunk number>

order by start, dbsname, tabname;

Replace chunk number with the number of the chunk for which you want to list the table names.