joinCache.pl

Summary
joinCache.pl
JOINCACHE tool
Variables
version
optionsContains joinCache call options :
thisInformations are treated, interpreted and store in this hash, using JOINCACHE classes :
Functions
mainMain method.
initChecks options and initializes the default logger.
configLoad all parameters from the configuration file, using JOINCACHE::PropertiesLoader.
checkParamsCheck is basic: sections have not to be empty
Validation methods
validateValidates all components, checks consistency.
validateSourcePyramidsFor each pyramid in composition, we create a JOINCACHE::SourcePyramid object and we control attributes (JOINCACHE::SourcePyramid::loadAndCheck), reading pyramid’s descriptor.
validateBboxesFor each bbox, we parse string to store values in array and we control consistency (min < max).
validateCompositionValidates each source in the composition.
Process methods
doItWe browse all source pyramids to identify images to generate.
Utils
searchImageSearch a tile in source pyramids with a lower priority.
tileToImageA pyramid’s image can contain several tiles.
Details
Command’s options
Write a configuration fileA file configuration can be composed of sections and parameters following :
JOINCACHE global configuration fileProperties by section.
logger
pyramidFor more informations about properties, see BE4::Pyramid.
process
bboxesWe define several bouding boxes and we attribute them an identifiant.
compositionWe say : “For this level, for this extent, I want to use images present in this source images’ pyramid”
Paths in brief
Externals tools

JOINCACHE tool

Synopsis

perl joinCache.pl --conf=path

Tool allowing to merge pyramids, using the same TMS.  Merge method (if an image is present in several source pyramids) can be :

  • replace : only the top pyramid’s image is kept
  • multiply : samples are multiplied
  • alphatop : alpha blending method
  • top : like replace, but more clever (we can avoid to remove data with nodata thanks to the masks)

Source pyramids could have different :

  • compression
  • number of samples per pixel

Source pyramids must have the same :

  • TMS
  • directory depth
  • sample format (unsigned 8-bit integer)
  • number of bits per sample
Summary
Variables
version
optionsContains joinCache call options :
thisInformations are treated, interpreted and store in this hash, using JOINCACHE classes :
Functions
mainMain method.
initChecks options and initializes the default logger.
configLoad all parameters from the configuration file, using JOINCACHE::PropertiesLoader.
checkParamsCheck is basic: sections have not to be empty
Validation methods
validateValidates all components, checks consistency.
validateSourcePyramidsFor each pyramid in composition, we create a JOINCACHE::SourcePyramid object and we control attributes (JOINCACHE::SourcePyramid::loadAndCheck), reading pyramid’s descriptor.
validateBboxesFor each bbox, we parse string to store values in array and we control consistency (min < max).
validateCompositionValidates each source in the composition.
Process methods
doItWe browse all source pyramids to identify images to generate.
Utils
searchImageSearch a tile in source pyramids with a lower priority.
tileToImageA pyramid’s image can contain several tiles.

Variables

version

options

my %options

Contains joinCache call options :

versionTo obtain the command’s version
helpTo obtain the command’s help
usageTo obtain the command’s usage
propertiesConfiguration file

this

my %this

Informations are treated, interpreted and store in this hash, using JOINCACHE classes :

propertiesLoaderJOINCACHE::PropertiesLoader - Contains all raw informations
pyramidBE4::Pyramid - Final pyramid’s components and tools
processJOINCACHE::Process - Compute the final pyramid
bboxesarray hash - Defines identifiants with associated validated bounding boxes (as array)
sourcePyramidsJOINCACHE::SourcePyramid hash - Key is the descriptor’s path.  We have an complete and without repetition list of source pyramids.
compositionhash - Contain validate and pre-treated composition :
level_id => priority => {
        extremTiles => [colMin, rowMin, colMax, rowMax],
        extremImages => [colMin, rowMin, colMax, rowMax], # = extremTiles / (number of tiles in the dimension)
        sourcePyramid => <JOINCACHE::SourcePyramid>
}
doneImagesboolean hash - To memorize already done tiles, we use this hash, containing “I_J => TRUE”.

Functions

main

sub main

Main method.

See Also

init, config, validate, doIt

init

sub init

Checks options and initializes the default logger.  Check properties file (mandatory).

config

sub config

Load all parameters from the configuration file, using JOINCACHE::PropertiesLoader.

See Also

checkParams

checkParams

sub checkParams

Check is basic: sections have not to be empty

Validation methods

validate

sub validate

Validates all components, checks consistency.  Use classes BE4::Pyramid, JOINCACHE::Process.

See Also

validateSourcePyramids, validateBboxes, validateComposition

validateSourcePyramids

sub validateSourcePyramids

For each pyramid in composition, we create a JOINCACHE::SourcePyramid object and we control attributes (JOINCACHE::SourcePyramid::loadAndCheck), reading pyramid’s descriptor.  TMS, tiles per width, tiles per height and directory depth have to be the same for every one.

validateBboxes

sub validateBboxes

For each bbox, we parse string to store values in array and we control consistency (min < max).

validateComposition

sub validateComposition

Validates each source in the composition.

  • levelId have to be in the TMS
  • bboxId have to be defined in the ‘bboxes’ section (and is interpreted)
  • used level have to be present in the pyramid

We store too pyramid’s format (phtomoetric, samples per pixel...).

Process methods

doIt

sub doIt

We browse all source pyramids to identify images to generate.

For each level, for each source pyramid :

Utils

searchImage

sub searchImage

Search a tile in source pyramids with a lower priority.

Parameters (list)

levelIdstring - Level in which we serach tiles
basePriorityinteger - Source’s priority from which we search tiles
i,jinteger - Searched tile’s indices
imagesarray reference - To store other found source images

tileToImage

sub tileToImage

A pyramid’s image can contain several tiles.  We convert tiles indices into image indices, dividing by tiles per width or tiles per height.

Parameters (list)

extremTilesarray - Extrem tiles, to convert into extrem images

Details

Summary
Command’s options
Write a configuration fileA file configuration can be composed of sections and parameters following :
JOINCACHE global configuration fileProperties by section.
logger
pyramidFor more informations about properties, see BE4::Pyramid.
process
bboxesWe define several bouding boxes and we attribute them an identifiant.
compositionWe say : “For this level, for this extent, I want to use images present in this source images’ pyramid”
Paths in brief
Externals tools

Command’s options

--helpDisplay the link to the technic documentation.
--usageDisplay the link to the technic documentation.
--versionDisplay the tool version.
--conf=pathPath to configuration file of the pyramid.  This option is mandatory.

Write a configuration file

A file configuration can be composed of sections and parameters following :

[ section ]
parameter = value ; comments
;comments

JOINCACHE global configuration file

Properties by section.

logger

log_levelDefine importance level from which we print messages.  From less to more important : DEBUG - INFO - WARN - ERROR - ALWAYS (WARN by default).

Example

[ logger ]
log_level = INFO

pyramid

For more informations about properties, see BE4::Pyramid.

pyr_name_newName of the new pyramid, used to name the descriptor, the pyramid’s list, the data directory and the temporary directory.
pyr_data_pathDirectory path, where to write the cache.  Mandatory.
pyr_desc_pathDirectory path, where to write the descriptor.  Mandatory.
compressionPossible values : raw, jpg, zip, lzw, png, pkb.  PNG compression is not an official compression for tiff format (just readable by rok4).  Default : raw.
colorThe color is a string and contain on value per sample, in decimal format, seperated by comma.  For 8 bits unsigned integer, value must be between 0 and 255.  Example : “255,255,255” (white).
image_width, image_heightNumber a tile in the cache image, widthwise and heightwise.  16 * 16 for example.  Mandatory.  Have to be the same in the source pyramids.
bitspersamplePossible values : 8, 32.  Have to be the same as source.  Mandatory.
sampleformatPossible values : uint, float.  Have to be the same as source.  Mandatory.
samplesperpixelPossible values : 1, 3, 4.  Have to be the same as source.  Mandatory.
photometricPossible values : gray, rgb.  Have to be consistent with the samples per pixel.  1 -> gray and 3,4 -> rgb.  Default : rgb.
dir_depthImage’s depth from the level directory. depth = 2 => /.../LevelID/SUB1/SUB2/IMG.tif.  Mandatory.  Have to be the same in the source pyramids.
dir_image, dir_nodata, dir_maskNames of subdirectories in the pyramid, in pyr_data_path/pyr_name/ Default : IMAGE, NODATA, MASK.
export_masksAn option is used to precise if we want to write masks in the final pyramid.  So we can use masks but not export them (they may be useless for a pyramid which wont be never updated).  Only values “true” and “TRUE” lead to masks’ export.  Optionnal, FALSE by default.
tms_nameTMS file name, with extension .tms.  Mandatory.  Have to be the same in the source pyramids.
tms_pathDirectory, where to find the TMS file.  Mandatory

Examples

[ pyramid ]
pyr_data_path = /home/IGN/PYRAMIDS
pyr_desc_path = /home/IGN/DESCRIPTOR
pyr_name_new = JOINED_PYRAMID

tms_name = LAMB93_10cm.tms
tms_path = /home/IGN/TMS

dir_depth = 2
image_width = 16
image_height = 16

dir_image = IMAGE
dir_nodata = NODATA
dir_mask = MASK
export_masks = TRUE

compression         = png
bitspersample       = 8
sampleformat        = uint
photometric         = rgb
samplesperpixel     = 3
interpolation       = bicubic

color               = 255,0,0 ; red as final nodata color

process

job_numberLevel of parallelization for scripts.
path_tempTemporary directory path proper to each script : temporary files are written in path_temp/pyr_name_new/SCRIPT_ID.  Mandatory.
path_temp_commonCommon temporary directory path : temporary files which have to be shared between several scripts are written in path_temp_common/pyr_name_new/COMMON.  Mandatory.
path_shellDirectory path, to write scripts in.  Scripts are named in like manner for all generation.  That’s why the path_shell must be specific to the generation (contains the name of the pyramid for example).  Mandatory.
merge_methodWay to merge several source pyramid’s images.  Possible values are : alphatop, replace, multiply, top.  See details in JOINCACHE::Process.
use_masksPrecise if we want to use masks.  Make heavier genrations.  Optionnal, FALSE if not provided.

Example

[ process ]
merge_method = alphatop
path_temp = /tmp/PRIVATE/
path_temp_common = /tmp/COMMON/
path_shell  = /home/IGN/SCRIPT/ORTHO
job_number = 16
use_masks = TRUE

bboxes

We define several bouding boxes and we attribute them an identifiant.  This identifiatn will be used in the composition section to precise the extent where the source pyramid is used.  If we use an undefined identifiant, an error will occure.

Format : ID = xmin,ymin,xmax,ymax , ID is a basic string, without space and coordinates are in the TMS’ SRS.

Example

[ bboxes ]
PARIS = 640000,6858000,658000,6867000
FXX = 0,6100000,1200000,7200000

composition

We say : “For this level, for this extent, I want to use images present in this source images’ pyramid”

Format : LevelID.BboxId = pyrPath1,pyrPath2 , BboxId have to be defined in bboxes section, LevelId have to be present in the TMS and in used pyramids.

Order in ‘composition’ is important, a triplet will take precedence over the next triplets for a same level.  We define priority like this.

Example

[ composition ]
16.PARIS = /home/IGN/PYRAMIDS/PARCEL_PNG_LAMB93_D075-O.pyr,/home/IGN/PYRAMIDS/ORTHO_RAW_LAMB93_D075-O.pyr

19.PARIS = /home/IGN/PYRAMIDS/PARCEL_PNG_LAMB93_D075-O.pyr
19.PARIS = /home/IGN/PYRAMIDS/ORTHO_RAW_LAMB93_D075-O.pyr

Paths in brief

  • Descriptor file : pyr_desc_path/pyr_name.pyr
  • List file: pyr_desc_path/pyr_name.list
  • Cache directory: pyr_data_path/pyr_name/
  • TMS file: tms_path/tms_name
  • Scripts: path_shell/SCRIPT_X.sh
  • Temporary directory: path_temp/pyr_name_new/SCRIPT_X
  • Common temporary directory: path_temp_common/pyr_name_new/COMMON

Externals tools

Scripts’ execution needs following tools

  • montage (imagemagick)
  • tiffcp (libtiff)
my %options
Contains joinCache call options :
my %this
Informations are treated, interpreted and store in this hash, using JOINCACHE classes :
sub main
Main method.
sub init
Checks options and initializes the default logger.
sub config
Load all parameters from the configuration file, using JOINCACHE::PropertiesLoader.
Reads a configuration file for joinCache, respect the IniFiles format, but consider order.
sub checkParams
Check is basic: sections have not to be empty
sub validate
Validates all components, checks consistency.
sub validateSourcePyramids
For each pyramid in composition, we create a JOINCACHE::SourcePyramid object and we control attributes (JOINCACHE::SourcePyramid::loadAndCheck), reading pyramid’s descriptor.
Store all informations about a pyramid.
sub loadAndCheck
We have to collect pyramid’s attributes’ values, parsing the XML file.
sub validateBboxes
For each bbox, we parse string to store values in array and we control consistency (min < max).
sub validateComposition
Validates each source in the composition.
sub doIt
We browse all source pyramids to identify images to generate.
sub searchImage
Search a tile in source pyramids with a lower priority.
sub tileToImage
A pyramid’s image can contain several tiles.
Store all informations about a pyramid.
Configure, assemble commands used to generate merged pyramid’s images, from source pyramids’ images.
sub treatImage
Close