obsFiles_mod

link to source code

Dependency Diagrams:

Direct Dependency Diagram

Reverse Dependency Diagram

Description

MODULE obsFiles_mod (prefix=’obsf’ category=’3. Observation input/output’)

Purpose

High-level module to handle reading/writing of observations that can be stored in one of several different formats. Currently, the only supported formats are:

1. BURP

2. SQLITE (burp2rdb format)

3. SQLITE (obsDB format)

Quick access

Variables

obsf_filename, obsf_nfiles

Routines

obsf_addcloudparametersandemissivity(), obsf_cleanobsfiles(), obsf_copyobsdirectory(), obsf_determinefiletype(), obsf_determinesplitfiletype(), obsf_filessplit(), obsf_getfilename(), obsf_obssub_read(), obsf_obssub_update(), obsf_readfiles(), obsf_setup(), obsf_setupfilenames(), obsf_updatemissingobsflags(), obsf_writeascidump(), obsf_writefiles(), obsf_writehx(), setheadbodyprimarykeycolumns(), setobsfilesmpiuniquelist()

Needed modules

• midasmpi_mod: MODULE midasMpi_mod (prefix=’mmpi’ category=’8. Low-level utilities and constants’)

• ramdisk_mod: MODULE ramDisk_mod (prefix=’ram’ category=’8. Low-level utilities and constants’)

• utilities_mod: MODULE utilities_mod (prefix=’utl’ category=’8. Low-level utilities and constants’)

• obsspacedata_mod: MODULE obsSpaceData_mod (prefix=’obs’ category=’6. High-level data objects’)

• burpfiles_mod: MODULE burpFiles_mod (prefix=’brpf’ category=’3. Observation input/output’)

• sqlitefiles_mod: MODULE sqliteFiles_mod (prefix=’sqlf’ category=’3. Observation input/output’)

• sqliteutilities_mod: MODULE sqliteUtilities_mod (prefix=’sqlu’ category=’3. Observation input/output’)

• obsdbfiles_mod: MODULE obsdbFiles_mod (prefix=’odbf’ category=’3. Observation input/output’)

• obsdiagfiles_mod: MODULE obsDiagFiles_mod (prefix=’diaf’ category=’3. Observation input/output’)

• bufr_mod: MODULE bufr_mod (prefix=’bufr’ category=’8. Low-level utilities and constants’)

• obssubspacedata_mod: MODULE obsSubSpaceData_mod (prefix=’oss’ category=’6. High-level data objects’)

• obsutil_mod: MODULE obsUtil_mod (prefix=’obsu’ category=’3. Observation input/output’)

• obsvariabletransforms_mod: MODULE obsVariableTransforms_mod (prefix=’ovt’ category=’4. Data Object transformations’)

• burpread_mod: MODULE burpRead_mod (prefix=’brpr’ category=’3. Observation input/output’)

• biascorrectionconv_mod: MODULE biasCorrectionConv_mod (prefix=’bcc’ category=’1. High-level functionality’)

• clibinterfaces_mod: MODULE clibInterfaces_mod (prefix=’’ category=’9. Global interfaces’)

• tovsnl_mod: MODULE tovsNL_mod (prefix=’tvs’ category=’5. Observation operators’)

• ensembleobservations_mod: MODULE ensembleObservations_mod (prefix=’eob’ category=’6. High-level data objects’)

Variables

• obsfiles_mod/obsf_filename (150) [character,public]

• obsfiles_mod/obsf_nfiles [integer,public]

Subroutines and functions

subroutine  obsfiles_mod/obsf_setup(datestamp_out, obsfilemode_in)

Arguments

• datestamp_out [integer ,out]

• obsfilemode_in [character ,in]

Called from

omf_ominusf(), omf_ominusfens(), midas_analysiserroroi, midas_diaghbht, midas_ensembleh, midas_gencoeff, midas_letkf, midas_obsimpact, midas_obsselection, midas_prepcma, midas_sstbias, midas_thinning, midas_var, midas_var1d

Call to

obsf_setupfilenames(), obsf_determinefiletype(), utl_abort(), brpf_getdatestamp(), odbf_getdatestamp(), sqlf_getdatestamp()

function  obsfiles_mod/obsf_filessplit()

Return

obsfilessplit_out [logical ]

Called from

inn_setupobs(), obsf_writefiles(), obsf_obssub_read(), obsf_obssub_update(), obsf_addcloudparametersandemissivity(), obsf_updatemissingobsflags(), obsf_copyobsdirectory(), midas_analysiserroroi, midas_diaghbht, midas_obsimpact, midas_prepcma, midas_var, midas_var1d

Call to

utl_abort()

subroutine  obsfiles_mod/obsf_readfiles(obsspacedata)

Arguments

obsspacedata [struct_obs ,inout]

Called from

inn_setupobs(), midas_prepcma, midas_thinning

Call to

utl_abort(), ram_fullworkingpath(), brpr_addelementstoburp(), bcc_biasactive(), obs_numheader(), obs_numbody(), brpf_readfile(), sqlf_readfile(), odbf_readfile(), setheadbodyprimarykeycolumns(), utl_isnamelistpresent(), obs_sethind()

subroutine  obsfiles_mod/obsf_writefiles(obsspacedata[, hxens_mpiglobal_opt[, ascidumpobs_opt[, writediagfiles_opt[, ensobs_opt]]]])

Arguments

obsspacedata [struct_obs ,inout]

Options

• hxens_mpiglobal_opt (*,*) [real ,in,]

• ascidumpobs_opt [logical ,in,]

• writediagfiles_opt [logical ,in,]

• ensobs_opt [struct_eob ,in,]

Called from

midas_analysiserroroi, midas_diaghbht, midas_letkf, midas_ominusf, midas_obsimpact, midas_obsselection, midas_prepcma, midas_thinning, midas_var, midas_var1d

Call to

utl_tmg_start(), utl_abort(), obsf_determinefiletype(), ovt_transformresiduals(), obsu_setassflg(), obsu_updatesourcevariablesflag(), obsf_determinesplitfiletype(), brpf_updatefile(), odbf_updatefile(), sqlf_updatefile(), obsf_writehx(), ram_getramdiskdir(), obsf_filessplit(), obsf_getfilename(), diaf_writeallsqldiagfiles(), obsf_writeascidump(), utl_tmg_stop()

subroutine  obsfiles_mod/obsf_cleanobsfiles()

Called from

midas_obsselection, midas_prepcma, midas_thinning

Call to

utl_abort(), obsf_determinefiletype(), utl_tmg_start(), obsf_determinesplitfiletype(), brpr_burpclean(), obdf_clean(), sqlf_cleanfile(), utl_tmg_stop()

subroutine  obsfiles_mod/obsf_writehx(obsspacedata, hxens_mpiglobal)

Arguments

• obsspacedata [struct_obs ,in]

• hxens_mpiglobal (*,*) [real ,in]

Called from

obsf_writefiles()

Call to

obs_numheader(), obs_write_hx()

subroutine  obsfiles_mod/obsf_writeascidump(obsspacedata)

Arguments

obsspacedata [struct_obs ,in]

Called from

obsf_writefiles()

Call to

obs_mpilocal(), obs_print()

subroutine  obsfiles_mod/obsf_setupfilenames()

Called from

obsf_setup()

Call to

ram_fullworkingpath(), obsf_determinesplitfiletype(), setobsfilesmpiuniquelist()

subroutine  obsfiles_mod/setobsfilesmpiuniquelist(basefilenamelist, filetypelist)

Purpose

Create a unique list of obs filenames/familyTypes across all mpi tasks.

Arguments

• basefilenamelist (*) [character ,in]

• filetypelist (*) [character ,in]

Called from

obsf_setupfilenames()

Call to

mmpi_allgather_string()

subroutine  obsfiles_mod/obsf_determinefiletype(obsfiletype)

Arguments

obsfiletype [character ,out]

Called from

obsf_setup(), obsf_writefiles(), obsf_cleanobsfiles()

Call to

utl_abort(), obsf_determinesplitfiletype()

subroutine  obsfiles_mod/obsf_determinesplitfiletype(obsfiletype, filename)

Arguments

• obsfiletype [character ,out]

• filename [character ,in]

Called from

obsf_writefiles(), obsf_cleanobsfiles(), obsf_setupfilenames(), obsf_determinefiletype(), obsf_obssub_read(), obsf_obssub_update(), obsf_addcloudparametersandemissivity(), obsf_updatemissingobsflags()

Call to

sqlu_sqltableexists(), utl_abort()

function  obsfiles_mod/obsf_getfilename(obsfam[, filefound_opt])

Purpose

Returns the observations file name assigned to the calling processor. If the input family has more than one file, the first file found will be returned.

Arguments

Arguments

obsfam [character ,in] :: observation family name :found_opt: logical indicating if a file could be found for the family (optional)

Options

filefound_opt [logical ,out,]

Return

filename [character ] :: file name of associated observations file

Called from

readoerfromobsfileforsw(), obsf_writefiles(), obsf_obssub_read(), obsf_obssub_update()

function  obsfiles_mod/obsf_obssub_read(obsfam, stnid, varno, nlev, ndim[, numcolumns_opt[, bkstp_opt[, block_opt[, match_nlev_opt[, codtyp_opt]]]]])

Purpose

Retrieves information for observations from observation files and returns the data in a struct_oss_obsdata object. Data will be retrieved for all nodes that have valid filenames for the specied observational family and combined into one struct_oss_obsdata if the observational files are split.

Arguments

Arguments

• obsfam [character ,in] :: observation family name

• stnid [character ,in] :: station ID of observation

• varno [integer ,in] :: BUFR code (if <=0, to search through all codes to obtain first between 10000 and 16000)

• nlev [integer ,in] :: number of levels in the observation

• ndim [integer ,in] ::

  number of dimensions for the retrieved data in

  each report (e.g. ndim=1 for std, ndim=2 for averaging kernel matrices)

  numColumns_opt

  Number of columns (if different from nlev and for ndim=2)

Options

• bkstp_opt [integer ,in,] :: bkstp number of requested block if BURP file type (optional)

• block_opt [character ,in,] :: block type of requested block if BURP file type (optional) Valid values are ‘DATA’, ‘INFO’, ‘3-D’, and ‘MRQR’, indicated by the two rightmost bits of bknat.

• match_nlev_opt [logical ,in,] :: determines if the report matching criteria includes checking if the report number of levels is the same as the input argument nlev (optional)

• codtyp_opt (*) [integer ,in,] :: optional CODTYP list for search (optional)

• numcolumns_opt [integer ,in,] :: Number of columns (if different from nlev and for ndim=2)

Return

obsdata [struct_oss_obsdata ] :: struct_oss_obsdata object

Called from

chm_read_obs_err_stddev(), oopc_readavgkern(), osd_readsqrthpht()

Call to

obsf_getfilename(), obsf_determinesplitfiletype(), utl_abort(), brpf_obssub_read(), obsf_filessplit(), oss_obsdata_alloc(), oss_obsdata_mpiallgather()

function  obsfiles_mod/obsf_obssub_update(obsdata, obsfam, varno[, bkstp_opt[, block_opt[, multi_opt]]])

Purpose

Add or modify data in observational files from data stored in a struct_oss_obsdata object.

Arguments

Arguments

• obsdata [struct_oss_obsdata ,inout] :: Input struct_oss_obsdata object for varno.

• obsfam [character ,in] :: observation family name

• varno (*) [integer ,in] :: BUFR descriptors. Number of elements must be max(1,obsdata%dim2)

Options

• bkstp_opt [integer ,in,] :: bkstp number of requested block if BURP file type (optional)

• block_opt [character ,in,] :: block type of requested block if BURP file type (optional) Valid values are ‘DATA’, ‘INFO’, ‘3-D’, and ‘MRQR’, indicated by the two rightmost bits of bknat.

• multi_opt [character ,in,] :: Indicates if intended report are for ‘UNI’ or ‘MULTI’ level data (optional)

Return

nrep_modified [integer ] :: Number of modified reports

Call to

obsf_getfilename(), obsf_filessplit(), obsf_determinesplitfiletype(), utl_abort(), brpf_obssub_update()

subroutine  obsfiles_mod/obsf_addcloudparametersandemissivity(obsspacedata)

Purpose

Loop on observation files to add cloud parameters and emissivity

Arguments

obsspacedata [struct_obs ,inout]

Called from

midas_obsselection

Call to

obsf_filessplit(), obsf_determinesplitfiletype(), sqlf_addcloudparametersandemissivity(), brpr_addcloudparametersandemissivity(), utl_abort()

subroutine  obsfiles_mod/obsf_updatemissingobsflags(obsspacedata)

Purpose

Loop on observation files to set missing observation flags to 2048 For now, this is done for only ATMS and AMSUA

Arguments

obsspacedata [struct_obs ,inout]

Called from

midas_obsselection

Call to

obs_getheaderindex(), obs_headelem_i(), tvs_isidburpinst(), obsf_filessplit(), obsf_determinesplitfiletype(), brpr_updatemissingobsflags()

subroutine  obsfiles_mod/obsf_copyobsdirectory(directoryinout, direction)

Purpose

Loop on observation files and copy each to and from the specified directory

Arguments

• directoryinout [character ,in]

• direction [character ,in]

Called from

midas_obsselection

Call to

obsf_filessplit(), utl_copyfile(), utl_abort()

subroutine  obsfiles_mod/setheadbodyprimarykeycolumns(obsdat, numheaderread, numbodyread)

Purpose

Set header/body primary keys in obsSpaceData that will ensure unique values over all mpi tasks.

Arguments

• obsdat [struct_obs ,inout]

• numheaderread [integer ,in]

• numbodyread [integer ,in]

Called from

obsf_readfiles()

Call to

obs_numheader(), obs_numbody(), obs_setheadprimarykey(), obs_setbodyprimarykey()