Research and development - WB auto -temperature correlation - tests - finding the optimum settings (#6643)

* Change estimchrom estim hue final

* Itcw observer checkbox

* 8 spectral colors

* 11 spectral colors

* 6 spectral colors

* Small change to correction green

* Fixed various bad behavior - settings itcwb_deltaspec

* 4 spectral colors green

* 3 green spectral colors

* 3 spectral colors green

* 5 spectral colors red green

* 3 spectral colors red green

* 3 spectral colrs green red

* Chnage slider choice deltaE - to size color patch

* display more info in console

* Display more info in console - comment code

* Clean - comment code

* 8 spectral colors green red

* 8 spectral colors blue

* 8 spectral colors blue

* 3 spectral colors

* 6 spectral colors

* 4 spectal colors

* 2 spectral colors

* Eliminate high Y - increse temp iteration - recenter selection inside patch

* compatibility 5.9

* Modify dislay and place selected dats in patch

* Dispaly in console hue chroma image selection

* First optimization

* optimization code - clean unused variables

* Remove sorted in chroma order - some others changes

* 5.9 compatibility

* Clean and optimize code

* Clean code - more display info in console

* More info in console - set itcwb_minsize in option

* Display patch chroma in GUI

* Change some default settings

* Minsize patch GUI

* Display GUI patch size

* Chroma patch variations

* Display in GUI min and max dats found in patch

* Some adjustments

* Remove too low datas in patch

* Various improvment - ponderate

* Clean code colortemp comment spectral datas

* Comment code - correc gren

* Comment and refine code

* Fixed bug in improcoordinator

* fixes a malfunction of AWB bias in improcoordinator

* Various improvments - clean code

* 2 improvments green and find patch

* Allows parametrind ponder and Ypurple in options

* Update autowb - filter on magenta purple

* Clean and optimize code

* Read colors in GUI

* Denoise median 5x5 image datas before Itcwb

* Change to GUI

* Replace median5x5 by 2 median3x3 - clean GUI code

* Work around to recalculate wbauto

* Remove work around - change params->wb related change

* Disabled low_sampling 5.9

* First try for compatibility Low sampling 5.9

* Re-enable WB autogrey

* Change reference image to autogrey if camera settings probably out

* Clean code aand adjust settings camera out

* Format astylert rawimagesource.cc - various small adjustments

* Small correction

* Small green correction

* Change default settings - correction to autowb bias in improccoordinator

* Save provisory work

* Save provisory work 2

* Try to find good tempitc with iterate when temptitc very different 5000K

* Change 2 default settings

* Work around awb temp bias

* Limit work around bias - small other changes

* 3 spectral colors deep blue

* 4 spectral color deep blue

* 4 color spectral yellow

* 5 spectral colors neutral red-magenta

* 5 spectral color neutral red magenta

* Change threshold tempitc

* 5.9 compatibility

* Work around bad claculation Planck's formula

* Fixed bad behavior when using blackbody spectal

* 6 spectral color - checkbutton remove 2 passes

* Fixed various bugs

* Format rawimagesource.cc

* Various improvment before to find the good selection

* First new optimization (perhaps?)

* Display GUI patc deltaE

* Various optimzation and tooltip

* Default observer 2°- Simplify GUI - improve display GUI - remove too low numbers datas

* 4 spectral colors - change 2 settings

* Change calculation threshold 2 passes

* 6 spectral colors magenta

* 13 spectral colors

* Improvment GUI display

* remove checkbutton no_purple

* 2 spectral colors - remove in GUI low sampling 5.9

* Change default setting display in console

* Change behavior when temp near stdA

* 4 spectral colors - settings Itcwb_tempstdA

* Gamut control on image histogram

* Clean code and format rawimagesource.cc

* Comment and format colortemp.cc

* Remove unused fileds from GUI - change tooltips

* Add tooltips

* Various changes

* Change settings when camera temperature is out

* improvment extra and tooltip

* Small change when using auto grey as start reference

* Remove itcwb_fgreen from GUI

* Various change - and 'unsatisfactory' test to take into account Custom

* Change second temp when < 4000 - replace automatic by automatic and refinement

* Neutralize 'Use custom tempertaure and green'

* Green refinement

* Clean code - small changes

* Other clean code

* Added forgotten itcw_green in procparams.cc

* Fixed a bug dispaly deltaE

* Change threshold settings - change code for Custom

* various change in case of using camera settings

* Change way to estimate range green

* clean code and message in console

* Clean and optimize

* Disable 2 passes if custom

* Improve GUI 2 passes

* Increase sampling and number of temperature above 9400K

* Hide AWB temperature bais and green refinement when Custom temerature ans green

* marks out the code with past time markers

* Improve time processing with spectral colors temp calculations

* Replace in console msec by nsec

* Scan temp 5.9

* Change max temp and behavior if temp > 15000k

* Refine temp near 14000K

* Increase sampling temperature 132 to 167

* Increase sampling temperature

* Set in preference choice for itcwb_custom

* White point calculated  with observer instead of table

* More on whitepoint calculation - informations

* Clean and format code with astyle colortemp.cc rawimagesource.cc improccoordinator.cc

* Small change tooltip

* Improve update auto-wb

* Remove 'use custom temperature and tin' - simplify sampling to 2 choice

* Small change to rand AWB bias - green refinement - tooltips

* slightly modify the parameters upstream of the algo - do not use camera in some cases

* Mixed initial green

* Refine init green and temp when no camera

* Refine temp and green in do not use camera settings

* Change tint threshold and various improvments (tooltips, wrong values..)

* Refinment and clean code

* Improve mode No use camera settings

* Init nocam

* Refine use no camera  settings

* Small change

* 5.9 compatibility - and format

* Refine sampling temperature  around 7000 - 8000K

* Format astylert rawimagesource colortemp

* 5.9 settings

* 5.9 settings 2

* Various change - 5.9 - format

* Improve pre and post algorithm

* Fixed correction temp > 8000K

* Fixed bad behavior Tppat.minchroma

* Change sampling choices and tooltip

* Change name wp wp2 wip to wb wb2 iwb to avoid misunderstood

* Small Change settings - limit display studgood to 0.0001

* Improvment display GUI temp

* Change default sampling

* Change ACESP0 to JDCmax

* Improve GUI abstract profile

* Allows dispaly in console xy values thanks to Reffort

* Remove bruce rgb in wp and abstract

* Change wp abstract and save itcwb with dispaly cellxy

* Change prim = 13 improccoordinator iplab2rgb

* remove display datas in console - fixed wrong display Y xyY

* Fixed ifferences in GUI maxhist minhist

* Change limits x y in function sampling

* fixed a dizziness

* Change  after algorithm rule when temp > 6000K

* Harmonize limx limy for oldsampling 5.9

* Added Cam xyz matrix to sampling - thanks to Reffort

* Display in console xyz matrix

* Forgotten instruction when mix camera and autowb grey

* Change rgbloc with Camera matrix Dcraw when no input profile

* Change rep to repref in rgbxyz

* Remove message in console

* 5.9 compat

* 5.9 compat 2

* 5.9 compat 3

* 5.9 compat 4

* 5.9 compat 5 format

* 5.9 compta 6

* Improve code

* Diasble itcwb_sampling in GUI

* Comment code

* Clean and comment code

* 5 spectral colors

* 9 spectral colors

* 2 spectral colors

* Various improvments

* 5 spectral colors

* 5 spectral colors

* First changes review lawrence37

* Second changes review Lawrence37

* 3rd changes review Lawrence37

* 4th changes review Lawrence37

* Change windows and appimage yml

* Re-enable Rtv4_Bruce in procparams.cc

* test for 5.9 compatibility

* Try to fixed crash

* Slightly improve the management of 5.9

* test for 5.9 compatibility by passing parameters via the GUI

* Forgotten parameter

* Change ref_spec limit and remove pac_start for itcwb_sampling

* Puts back the old Planck formula - the same as in dev

* Revert "test for 5.9 compatibility by passing parameters via the GUI"

This reverts commit d6b5a8feda31b93a5574cee6ebf4d3e9d58bbcfb.

* Switch to new ITCWB only when changing WB method

Prevents sudden changing of the white balance if ITCWB is selected and
the image is reopened.

* Fix spelling in comment

* Suggested change from Lawrence37 to whitebalance - small change to improcoordinator

* tempbias Set_sentive(false) if itcwb_smapling - clean whitebalance

* Another clean whitebalance.cc

---------

Co-authored-by: Lawrence Lee <45837045+Lawrence37@users.noreply.github.com>

rtdata/languages/default

@@ -1535,6 +1535,22 @@ HISTORY_MSG_WAVTHRDEN;Threshold local contrast
 HISTORY_MSG_WAVTHREND;Threshold local contrast
 HISTORY_MSG_WAVUSHAMET;Clarity method
 HISTORY_MSG_WBALANCE_OBSERVER10;Observer 10°
+HISTORY_MSG_WBITC_OBS;Remove algo 2 passes
+HISTORY_MSG_WBITC_CUSTOM;Itcwb Custom
+HISTORY_MSG_WBITC_SAMPLING;Low sampling
+HISTORY_MSG_WBITC_THRES;Itcwb Theshold
+HISTORY_MSG_WBITC_NOPURPLE;Itcwb Nopurple
+HISTORY_MSG_WBITC_PRIM;Primaries
+HISTORY_MSG_WBITC_SORTED;Itcwb ponderated
+HISTORY_MSG_WBITC_FORCE;Itcwb Force
+HISTORY_MSG_WBITC_SIZE;Itcwb Size
+HISTORY_MSG_WBITC_PRECIS;Itcwb Precision
+HISTORY_MSG_WBITC_DELTA;Itcwb Delta green
+HISTORY_MSG_WBITC_FGREEN;Itcwb Green - student
+HISTORY_MSG_WBITC_RGREEN;Itcwb Green range
+HISTORY_MSG_WBITC_MINSIZE;Patch min size
+HISTORY_MSG_WBITC_PONDER;Itcwb ponderated
+HISTORY_MSG_WBITC_GREEN;Grren refinement
 HISTORY_NEWSNAPSHOT;Add
 HISTORY_NEWSNAPSHOT_TOOLTIP;Shortcut: <b>Alt-s</b>
 HISTORY_SNAPSHOT;Snapshot
@@ -1841,6 +1857,17 @@ PREFERENCES_CHUNKSIZE_RGB;RGB processing
 PREFERENCES_CIE;Ciecam
 PREFERENCES_CIEARTIF;Avoid artifacts
 PREFERENCES_WBA;White Balance
+PREFERENCES_WBACORR;White Balance - Automatic temperature correlation
+PREFERENCES_WBACORR_TOOLTIP;These settings allow, depending on the images (type of raw file, colorimetry, etc.), an adaptation of the " Temperature correlation " algorithm in order to obtain the best overall results. There is no absolute rule, linking these parameters to the results obtained.\n\nThe settings are of 3 types: \n* those accessible to the user from the GUI.\n* those accessible only in reading from each pp3 file : Itcwb_minsize=20, Itcwb_delta=4 Itcwb_rgreen=1 Itcwb_nopurple=false (See Rawpedia)\n* those accessible to the user in 'options' (see Rawpedia)\n You can use "Awb temperature bias" and "Green refinement" to adjust the results. Each movement of these commands brings a new calculation of temperature, tint and correlation.\n\nPlease note that the 3 indicators 'Correlation factor', 'Patch chroma' and ΔE are given for information only. It is not because one of these indicators is better that the result will necessarily be better.
+PREFERENCES_WBAENA;Show White Balance auto Temperature correlation Settings
+PREFERENCES_WBAENACUSTOM;Use Custom temperature & tint
+PREFERENCES_WBAPATCH;Number maximum of colors used in picture
+PREFERENCES_WBAFORC;Forces Extra algoritm
+PREFERENCES_WBAGREENDELTA;Delta temperature in green iterate loop (if Force Extra enabled)
+PREFERENCES_WBANOPURP;No purple color used
+PREFERENCES_WBAPRECIS;Precision algorithm - scale used
+PREFERENCES_WBASIZEREF;Size of reference color compare to size of histogram color
+PREFERENCES_WBASORT;Sort in chroma order instead of histogram 
 PREFERENCES_CLIPPINGIND;Clipping Indication
 PREFERENCES_CLUTSCACHE;HaldCLUT Cache
 PREFERENCES_CLUTSCACHE_LABEL;Maximum number of cached CLUTs
@@ -2616,6 +2643,7 @@ TP_ICM_WORKING_PRIM_BRU;BruceRGB
 TP_ICM_WORKING_PRIM_BST;BestRGB
 TP_ICM_WORKING_PRIM_CUS;Custom (sliders)
 TP_ICM_WORKING_PRIM_CUSGR;Custom (CIE xy Diagram)
+TP_ICM_WORKING_PRIM_JDCMAX;JDC Max
 TP_ICM_WORKING_PRIM_NONE;Default
 TP_ICM_WORKING_PRIM_PROP;ProPhoto
 TP_ICM_WORKING_PRIM_REC;Rec2020
@@ -4077,7 +4105,7 @@ TP_WAVELET_WAVOFFSET;Offset
 TP_WBALANCE_AUTO;Auto
 TP_WBALANCE_AUTOITCGREEN;Temperature correlation
 TP_WBALANCE_AUTOOLD;RGB grey
-TP_WBALANCE_AUTO_HEADER;Automatic
+TP_WBALANCE_AUTO_HEADER;Automatic & Refinement
 TP_WBALANCE_CAMERA;Camera
 TP_WBALANCE_CLOUDY;Cloudy
 TP_WBALANCE_CUSTOM;Custom
@@ -4104,6 +4132,47 @@ TP_WBALANCE_FLUO_HEADER;Fluorescent
 TP_WBALANCE_GREEN;Tint
 TP_WBALANCE_GTI;GTI
 TP_WBALANCE_HMI;HMI
+TP_WBALANCE_ITCWB_DELTA;Delta temperature in green loop
+TP_WBALANCE_ITCWBDELTA_TOOLTIP;Fixed for each "green" iteration tried, the temperature difference to be taken into account.
+//TP_WBALANCE_ITCWB_FORCED;Forces use of the entire CIE diagram
+TP_WBALANCE_ITCWB_FORCED;Close to full CIE diagram
+TP_WBALANCE_ITCWFORCED_TOOLTIP;By default (box not checked) the data scanned during sampling is brought back to the sRGB profile, which is the most widespread, both for calibrating DCP or ICC profiles with the Colorchecker24, or used on the web.\n If you have very high gamut images (some flowers, artificial colors), then it may be necessary to use the entire CIExy diagram, the profile used will be ACESP0. In this second case, the number of colors that can be used in internal to the algorithm will be more important.
+TP_WBALANCE_ITCWB_FGREEN;Find green student
+TP_WBALANCE_ITCWGREEN;Green refinement
+TP_WBALANCE_ITCWGREEN_TOOLTIP;Allows you to change the "tint" (green) which will serve as a reference when starting the algorithm. It has substantially the same role for greens as "AWB temperature bias" for temperature.\nThe whole algorithm is recalculated.
+TP_WBALANCE_ITCWBFGREEN_TOOLTIP;Find the best compromise between Student and green.
+TP_WBALANCE_ITCWB_FRA;Auto Temperature correlation Settings
+TP_WBALANCE_ITCWB_FRA_TOOLTIP;These settings allow, depending on the images (type of raw, colorimetry, etc.), an adaptation of the 'Temperature correlation' algorithm. There is no absolute rule linking these parameters to the results obtained.
+TP_WBALANCE_ITCWB_MINSIZEPATCH;Patch minimum size
+TP_WBALANCE_ITCWBMINSIZEPATCH_TOOLTIP;Allows you to set the minimum patch value. values that are too low can lead to a lack of correlation.
+TP_WBALANCE_ITCWB_NOPURPLE;Filter on purple color
+//TP_WBALANCE_ITCWBNOPURPLE_TOOLTIP;By default when "Inpaint opposed" is activated, purple colors are not taken into account. However, if the image does not need highlight reconstruction, or if this image naturally contains purple tints (flowers, etc.), it may be necessary to deactivate, to take into account all the colors.
+TP_WBALANCE_ITCWBNOPURPLE_TOOLTIP;Allows you to filter magenta/purple data from the image. If the box is checked a filter limiting the value of Y is applied. By default this value is 0.4. You can change it in 'options' Itcwb_Ypurple (Maximum 1)
+TP_WBALANCE_ITCWB_ALG;Remove 2 pass algorithm
+TP_WBALANCE_ITCWB_CUSTOM;Use Custom temperature & tint
+TP_WBALANCE_ITCWCUSTOM_TOOLTIP;Allows you to use Custom settings Temperature and Green (tint).\n\nUsage tips:\n1) start Itcwb , enable 'Use Custom temperature and tint'.\n2) Set 'Temperature and tint' to your liking :free, Pick,...(Custom)\n3) go back to 'Temperature correlation'.\n\nYou cannot use : 2 passes, AWB temperature bias, Green refinement.
+TP_WBALANCE_ITCWALG_TOOLTIP;Allows you to switch to the other Alternative temperature (Alt_temp), when possible.\nInactive in the "single choice" case.
+TP_WBALANCE_ITCWB_PRECIS;Precision algorithm - scale used
+TP_WBALANCE_ITCWBPRECIS_TOOLTIP;The lower the value, the more relevant the data, but increases the processing time. Since the processing time is low, this parameter should generally be able to remain at the default value
+TP_WBALANCE_ITCWB_PRIM_SRGB;Low sampling & No use Camera settings
+TP_WBALANCE_ITCWB_PRIM_ADOB;Medium sampling
+TP_WBALANCE_ITCWB_PRIM_BETA;Medium sampling - near Pointer's gamut
+TP_WBALANCE_ITCWB_PRIM_REC;High sampling
+TP_WBALANCE_ITCWB_PRIM_ACE;Forces use of the entire CIE diagram
+TP_WBALANCE_ITCWB_PRIM_JDCMAX;Close to full CIE diagram
+TP_WBALANCE_ITCWB_PRIM_XYZCAM;Camera XYZ matrix
+TP_WBALANCE_ITCWB_PRIM_XYZCAM2;JDCmax after Camera XYZ matrix
+TP_WBALANCE_ITCWPRIM_TOOLTIP;Allows you to select the image sampling.\n'Close to full CIE diagram' almost uses the data present on the sensor, possibly including the imaginary colors.\n'Camera XYZ matrix' - uses the matrix directly derived from Color Matrix.\n'Medium sampling' (default) - near Pointer's gamut: corresponds substantially to the most common cases of human vision.\nThe other choice 'Low sampling and No use camera settings' allow you to isolate high gamut parts of the image and forces the algorithm in some cases (tint > 0.8,...) not to use camera settings. This will obviously have an impact on the result.\n\nThis sampling only has an influence on the channel multipliers, it has nothing to do with the "working profile" and does not modify the gamut of the image.
+TP_WBALANCE_ITCWB_RGREEN;Green range
+TP_WBALANCE_ITCWBRGREEN_TOOLTIP;Sets the green value review amplitude in iterations, from low amplitude 0.82 to 1.25 to maximum amplitude 0.4 to 4.
+TP_WBALANCE_ITCWB_SAMPLING;Low sampling 5.9
+TP_WBALANCE_ITCWSAMPLING_TOOLTIP;Allows you to use the old sampling algorithm to ensure better compatibility with 5.9. You must enable Observer 10° (default).
+TP_WBALANCE_ITCWB_SIZE;Size of ref. color compare to histogram
+TP_WBALANCE_ITCWB_SIZEPATCH;Size of color patch
+TP_WBALANCE_ITCWBSIZEPATCH_TOOLTIP;This setting sets the size of color datas used by algorithm.
+TP_WBALANCE_ITCWBSIZE_TOOLTIP;This setting sets the number of iterations to find the best correspondence between the reference spectral colors and those in xyY value of the image. A value of 3 seams a good compromise.
+TP_WBALANCE_ITCWB_THRES;Colors used in picture (preset)
+TP_WBALANCE_ITCWBTHRES_TOOLTIP;Limits comparison sampling between spectral data and image data.
 TP_WBALANCE_JUDGEIII;JudgeIII
 TP_WBALANCE_LABEL;White Balance
 TP_WBALANCE_LAMP_HEADER;Lamp
@@ -4115,6 +4184,10 @@ TP_WBALANCE_MULLABEL;Multipliers: r=%1 g=%2 b=%3
 TP_WBALANCE_MULLABEL_TOOLTIP;Values given for information purposes. You cannot change them.
 TP_WBALANCE_OBSERVER10;Observer 10° instead of Observer 2°
 TP_WBALANCE_OBSERVER10_TOOLTIP;The color management in Rawtherapee (White balance, channel multipliers, highlight recovery,...) uses the spectral data of the illuminants and colors. Observer is an important parameter of this management which takes into account the angle of perception of the eye. In 1931 it was fixed at 2° (privileges the use of the cones). In 1964 it was fixed at 10° (privileges the use of the cones, but partially takes into account the rods).\nTo avoid a (rare) drift of the colors due to the choice Observer 10° - probably due to the conversion matrix - Observer 2° must be selected.\nIn a majority of cases Observer 10° (default) will be a more relevant choice.
+TP_WBALANCE_PATCHLABEL;Read colors:%1 Patch: Chroma:%2 Size=%3
+TP_WBALANCE_PATCHLABEL_TOOLTIP;Display number of read colors (max=237).\nDisplay calculated Patch Chroma.\nAWB temperature bias, lets try to reduce this value, a minimum may seem to optimize the algorithm.\n\nPatch size matching chroma optimization.
+TP_WBALANCE_PATCHLEVELLABEL;Patch: ΔE=%1 - datas x 9 Min:%2 Max=%3
+TP_WBALANCE_PATCHLEVELLABEL_TOOLTIP;Display ΔE patch (this assumes there is enough spectral data), between image and spectral datas.\n Display read datas found. The 2 values correspond to the minimum and maximum data values taken into account. The coefficient x9 must be taken into account to obtain the number of pixels concerned in the image.
 TP_WBALANCE_PICKER;Pick
 TP_WBALANCE_SHADE;Shade
 TP_WBALANCE_SIZE;Size:
@@ -4123,10 +4196,12 @@ TP_WBALANCE_SOLUX41;Solux 4100K
 TP_WBALANCE_SOLUX47;Solux 4700K (vendor)
 TP_WBALANCE_SOLUX47_NG;Solux 4700K (Nat. Gallery)
 TP_WBALANCE_SPOTWB;Use the pipette to pick the white balance from a neutral patch in the preview.
-TP_WBALANCE_STUDLABEL;Correlation factor: %1
-TP_WBALANCE_STUDLABEL_TOOLTIP;Display calculated Student correlation.\nLower values are better, where <0.005 is excellent,\n<0.01 is good, and >0.5 is poor.\nLow values do not mean that the white balance is good:\nif the illuminant is non-standard the results can be erratic.\nA value of 1000 means previous calculations are used and\nthe resultsare probably good.
+TP_WBALANCE_STUDLABEL;Correlation factor: %1 Passes:%2 Worst_alt=%3
+TP_WBALANCE_STUDLABEL1;Correlation factor: %1 Passes:%2 Best_alt=%3
+TP_WBALANCE_STUDLABEL0;Correlation factor: %1 Passes:%2 Alt=%3
+TP_WBALANCE_STUDLABEL_TOOLTIP;Display calculated Student correlation.\nLower values are better, where <0.005 is excellent,\n<0.01 is good, and >0.5 is poor.\nLow values do not mean that the white balance is good:\nif the illuminant is non-standard the results can be erratic.\nA value of 1000 means previous calculations are used and\nthe resultsare probably good.\n\nPasses : number of passes made.\nAlt_temp : Alternative temperature.
 TP_WBALANCE_TEMPBIAS;AWB temperature bias
-TP_WBALANCE_TEMPBIAS_TOOLTIP;Allows to alter the computation of the 'auto white balance'\nby biasing it towards warmer or cooler temperatures. The bias\nis expressed as a percentage of the computed temperature,\nso that the result is given by 'computedTemp + computedTemp * bias'.
+TP_WBALANCE_TEMPBIAS_TOOLTIP;Allows to alter the computation of the 'auto white balance'\nby biasing it towards warmer or cooler temperatures. The bias\nis expressed as a percentage of the computed temperature,\nso that the result is given by 'computedTemp + computedTemp * bias'.\n\nYou can use "Awb temperature bias" to adjust the "Temperature correlation" results. Each movement of this command brings a new calculation of temperature, tint and correlation.
 TP_WBALANCE_TEMPERATURE;Temperature
 TP_WBALANCE_TUNGSTEN;Tungsten
 TP_WBALANCE_WATER1;UnderWater 1