Do more in less time!   TK-MIP helps guide you to do it correctly the VERY first time!      By clicking on the above buttons, representative TK-MIP GUI screens can be accessed as viewable examples to verify our clear, intuitive, user-prompting style.

(Our navigation buttons are at the TOP of each screen.)

Get free TK-MIP® tutorial software that demonstrates TeK Associates’ software development style.

Microsoft Word™ is to WordPerfect™ as TK-MIP™ is to ... everything else that claims to perform comparable processing!

    TeK Associates’ primary software product for the PC is TK-MIPTM, which encapsulates both historical and recent statistical estimation and Kalman filter developments and performs the requisite signal processing intuitively, accurately, and efficiently for easily accessible I/O results that are sought.

Harness the strength and power of a polymath to benefit you! It’s encapsulated within TK-MIP!

Also, somewhat surprisingly, for certain Civil & Environmental Engineers (see References [80], [81] at bottom of this screen).

Also see Ref. [125] offering use of Kalman Filter technology to Economics-related Fields (i.e., econometrics).

The reason why we have the delimiter of "some" preceding certain potential customers or clients in the above list is to be truthful in our list when we know that fields of specialization are so vast that, clearly, not all physicists nor all applied mathematicians, nor all statisticians would be concerned with the estimation aspects that we offer. We feel that the situation is similar for students in these respective areas for the same reasons. Cosmologists may not care. Particle Physicists may not care. Astronomers may care because fundamental estimation techniques harken back and originated with Gauss himself using least squares batch to track the orbits of planets in our solar system. Geologist may not care. Paleontologist may not care. If we did NOT recognize these facts and properly delimit potential USERS, it would constitute false advertising and indicate that we were unaware of what they do care about. Many statisticians can work their statistical magic on the outputs of Kalman Filters since they may have many underlying considerations in common. E.g., consider Price's theorem: https://en.wikipedia.org/wiki/Price_equation. There is another Price's theorem that arises when working with multidimensional Gaussians: Its use also arises when deriving the theory for a scalar level-crossing statistic for a binary hypothesis test; so that when the test statistic crosses above a decision threshold, then the conclusion is drawn between: H₁ vs. H_o (Hypothesis for Situation 1 or Event 1 holding or existing versus the Null Hypothesis, for Situation 0 or Event 0 holding or existing).

Question again: “Who Needs TK-MIP?” Answer: “Anyone with a need to either:..”

The above capabilities of our very USER-centric TK-MIP® should be of high interest to potential customers because:

1. Kalman Filters (KF) are in widespread use and occupy a prominent role in a variety of applications that reflect how modern technology has evolved in the last 50+ years. (As concisely summarized by us as a 20 item addendum [best read using the chronological option, which is opposite to the default being last-as-first; merely click on phrase "New Comments First" to expose the two hidden user/reader options, then choose/click on "Old Comments First"] in:  http://blogs.mathworks.com/headlines/2016/09/08/this-56-year-old-algorithm-is-key-to-space-travel-gps-vr-and-more/) Our pertinent comments were removed from this particular MathWork's Blog on 11 May 2021 and did not appear there for another week. I suspect that I had probably overstepped my welcome. Sorry!; It now appears there again; however, it can always be found by clicking here.

2. For KF applications: Linear Time Invariant (LTI) case <=> EASY!

3. For KF applications: Nonlinear or Time-varying cases <=> HARD!

4. Real-world applications are almost always nonlinear and time-varying!

5. For KF & LQG/LTR: LTI is what students learn and practice on in school.

6. Many competitor’s have capable, young, energetic employees with excellent computer skills. However, few are familiar with the above reality!

7. TK-MIP handles Nonlinear & Time-Varying (without pain). TK-MIP was developed in the background, between client projects, over 20+ years by TeK Associates personnel that are intimately familiar with both the theory and its relevant applications (and their respective goals, and with writing out the solutions and then coding them up).

8. TK-MIP handles nonlinear & time-varying application situations right “out-of-the-box”.

9. USER must set obvious initial switches within TK-MIP from dropdown menu to match architecture needed for specific application being tackled!

10. No programming is necessary to use and run TK-MIP (yet some customization of a sort is availed, if needed, but restricted to specific locations outside of the primary TK-MIP kernel [which has already been validated and verified by TeK Associates as performing correctly] of otherwise immutable TK-MIP software structure relegated to being merely: Input/Output (I/O) types and associated baud or sample rates & rotational and bias offset transformations invoked [all possibly time-varying]). These restrictions are enforced to prevent Users from inadvertently clobbering existing coded algorithms within TK-MIP that have already been validated by TeK Associates as performing properly.

11. TK-MIP also handles any algebraic loops encountered in the feedback path with finesse rather than with brute force (as otherwise encountered if USER needs to invoke Gear’s Integration for state or estimator propagation in time).

12. Estimation applications involving navigation systems frequently simplify to use an approximation of uniform time steps and periodic sensor measurements within simulation studies. Real-world navigation applications are more likely to be asynchronous (unless each measurement source is sampled periodically, as triggered by an accurate time clock used in common for each). Active Radar and sonar applications are always asynchronous since measurement receipt depends upon the distance from the target, which can vary continuously as own platform location and target location vary and changing environmental conditions can also affect this situation, as does the pulse repetition frequency [prf]). TK-MIP possesses the capability of routinely handling all these aforementioned situations.

13. Click here to view a discussion for specialists on the topic of Matrix Positive Semi-definiteness.   Click here to view a discussion for specialists on the topic of warnings about numerical tests for Matrix Positive Semi-definiteness.

14. On 19 February 2008, Rudolf F. Kalman (now deceased in 2016) received the prestigious $500,000 Charles Stark Draper Prize in Washington, D. C. for his original development or derivations consisting of 3 papers, one published in 1959 and the other two in the early 1960's. It is finally universally recognized that these algorithms now permeate and revolutionized all of modern technology (although first recognized early on with regard to weapons systems and national defense).

The graphic image screens CENTERED sequentially below are representative sample excerpts from TeK Associates' existing TK-MIP software for
the PC (that provide auxiliary information beyond the primary information conveyed by clicking the BUTTONS at the TOP of our PRODUCTS screen above):

Update: Now 50+ years of experience with Kalman Filter applications and he is an IEEE Life Senior Member and an Associate Fellow of the AIAA in GNC and a Member of SPIE. https://spie.org/profile/Thomas.KerrIII-2982?SSO=1 

An option is that the reader may further pursue any of the underlined topics presented here at their own volition merely by clicking on the underlined links that follow next. We offer a detailed description of our stance on use of State Variables and Descriptor representations and our strong apprehension concerning use of the Matrix Signum Function and on MatLab’s apparent mishandling of Level Crossing situations and our view of existing problems with certain Random Number Generators (RNG’s) and other Potentially Embarrassing Historical Loose Ends further below. These particular viewpoints motivated the design of our TK-MIP software to avoid these particular problems that we are aware of and also seek to alert others to. We are cognizant of historical state-of-the-art software as well [39]-[42]. At General Electric Corporate Research & Development Center in Schenectady, NY starting in 1971, Dr. Kerr was a protégée of his fellow coworkers: Joe Watson, Hal Moore, Dr. Glenn Roe, Dean Wheeler, Joel Fleck, Peter Meenan, and Ernie Holtzman within Dr. Richard Shuey's 5046 Industrial Simulations Group performing industrial modeling and computer simulation and analysis in other computational aspects related to the Automated Dynamic Analyzer (ADA) continuous-discrete digital simulation of Gaussian white noises as they arise within ODE integration in feed-forward and feedback loops involving some active but predominately passive elements.

Please click on Summary & Pricing button at top of this screen for a free demo download representative of our TK-MIP® software. If any potential customer has further interest in purchasing our TK-MIP® software, a detailed order form for printout (to be sent back to us via mail or fax) is available within the free demo by clicking on the obvious Menu Item appearing at the top of the primary demo screen (which is the Tutorials Screen within the actual TK-MIP® software). We also include representative numerical algorithms (fundamental to our software) for users to test for numerical accuracy and computational speed to satisfy themselves regarding its efficacy and efficiency before making any commitment to purchase. We have followed the development of this optimal and sub-optimal estimation technology for over 50 years and continue to do so with eternal vigilance so that TK-MIP users can immediately reap the benefits of our experience and expertise (without having to be experts in these areas themselves). Early on, we availed ourselves with the wisdom from the following technical literature in this area (that we also acquired, read thoroughly, and summarize herein where needed):

Comment: Dr. Eli Brookner's (Raytheon-retired) book, listed as #14 above, provides its very nice Chapter 3 that is especially helpful in handling the important Kalman filter applications to radar. He not only provides state-of-the-art but also filter state selection designs that were historically relied upon when computer capabilities were more restricted than today and designers were forced to simplify and rely on mere alpha-beta filters (corresponding to position and assumed constant velocity in a Kalman filter in as many dimensions as are actually being considered by the associated radar application: 2-D or 3-D), or on mere alpha-beta-gamma filters (corresponding to position and velocity and assumed constant acceleration in a Kalman filter in as many dimensions as are actually being considered by the associated radar application: 2-D or 3-D). Other historical conventions such as g-h filters or g-h-k filters and common variations such as that of "Benedict-Bordner" are also explained along with their appropriate track initiation. The relation between Kalman filters and Weiner Filters is also addressed, similar to what was provided in [95] (i.e., #6 in the above list). 

[The above historical perspective is important when the software algorithms present in older hardware are to be upgraded within newer hardware to enable enhanced capabilities.]

#30 as an addendum to the above list: Candy, J. V., Model-Based Signal Processing, Simon Haykin (Editor), IEEE Press and Wiley-Interscience, A John Wiley & Sons, Inc. Publication, Hoboken, NJ, 2006.

#31 as an addendum: Bruce P. Gibbs, ADVANCED KALMAN FILTERING, LEAST-SQUARES AND MODELING: A Practical Handbook, John Wiley & Sons, Inc., Hoboken, New Jersey, 2011. 

#32 as an addendum: Yaakov Bar-Shalom and William Dale Blair (Eds.), Multitarget-Multisensor Tracking: Applications and Advances," Vol. III, Artech House Inc., Boston, 2000.
Blair and Keel have a nice concise overview of radar system considerations for tracking in Chapt. 7 but their system dynamics model is ONLY linear. Notable is a nice section (Chapt. 8) on Countermeasure considerations and how they specifically affect
tracking simulations and implementations. This and the previously cited radar system considerations motivated my purchase of this book (especially ECM and ECCM of Tables 8.1 to 8.3).

https://ntrs.nasa.gov/api/citations/19770015864/downloads/19770015864.pdf  (Yes, Tom Kerr was there in person, as can be verified on page 209 of the attendance list at the end.)

For more analytical background perspectives of the entire field, please click on this.

For a system to be Stabilizable [108], those states that are NOT Controllable must decay to 0 anyway.  

For a system to be Detectable [108], those states that are NOT Observable must decay to 0 anyway.
From our practical experience, we know that a Kalman Filter can frequently still work satisfactorily even when systems are neither both Controllable and Observable nor both Stabilizable and Detectable.

However, without "Observability and Controllability" both holding, the rate of convergence of a Kalman Filter is much slower (sometimes painfully slower) 
than being asymptotically exponentially fast. A fully compliant Kalman Filter application does possess or exhibit asymptotically exponentially fast 
convergence of both the covariances and corresponding estimator that utilizes these associated covariances (and analytic proofs of this were provided by Kalman himself in 1960 and by others later).
 R. E. Kalman and J. E. Bertram, "Control System Analysis and Design Via the 'Second Method' of Lyapunov. Part I. Continuous Time Systems," 
Journal of Basic Engineering, Transactions of ASME, series D, Vol. 82, pp. 371-393, 1960. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.361.6851&rep=rep1&type=pdf 

Please click this for a simple overview perspective. 

For more information on the above algorithms, please see [109] and [133].

The numbers computationally obtained are consistent with what has historically first been rigorously proved analytically using proper mathematical analysis and statistical analysis.

TeK Associates® is currently offering a high quality Windowsä 9x\ME\WinNT \2000\XP\Vista\Windows 7\Windows 10 (see Note in Reference [1] within REFERENCES at the bottom of this web page) compatible and intuitive menu-driven PC software product TK-MIP for sale (to professional engineers, scientists, statisticians, and mathematicians and to the students in these respective disciplines) that performs Kalman Filtering for applications possessing linear models and exclusively white Gaussian noises with known covariance statistics (and can perform many alternative approximate nonlinear estimation variants for practically handling nonlinear applications) as well as performing Kalman Smoothing\Monte-Carlo System Simulation and (Optionally) Linear Quadratic Gaussian\Loop Transfer Recovery (LQG\LTR) Optimal Feedback Regulator Control for ONLY the LTI case and which also provides:

·    Clear on-line tutorials in the supporting theory, including explicit block diagrams describing TK-MIP processing options,

·    a clear, self-documenting, intuitive Graphical User Interface (GUI). No USER manual is needed. This GUI is easy to learn; hard to forget; and built-in prompting is conveniently at hand as a reminder for the USER who needs to use TK-MIP only intermittently (as, say, with a manager or other multi-disciplinary scientist or engineer).

·      a synopsis of significant prior applications (with details from our own pioneering experience),

·      use of Military Grid Coordinates (a.k.a. World Coordinates) for results as well as being in standard Latitude, Longitude, and Elevation coordinates for input and output object and sensor locations, both being available within TK-MIP (for quick and easy conversions back and forth), and for possible inclusion in map output,

·        implementation considerations and a repertoire of test problems for on-line software calibration\validation,

·       use of a self-contained on-line consultant feature which includes our own TK-MIP Textbook and also provides answers and solutions to many cutting edge problems in the specific topic areas,

·       how to access our 1000+ entry bibliography of recent (and historical) critical technological innovations that are included in a searchable database (directly accessible and able to be quickly searched  using any derived keyword present in the reference citation, e.g., GPS, author’s name, name of conference or workshop, date, etc.),

·       option of invoking Jonker-Valgenant-Castanon (J-V-C) algorithm for solving the inherent “Assignment Problem” of Operations Research that arises within Multi-Target Tracking (MTT) utilizing KF-like estimators,

·       compliance with Department of Defense (DoD) Directive 8100.2 for software, 

·        and manner of effective and efficient TK-MIP® use, 

so Users can learn and progress at their own rate (or as time allows) with a quick review always being readily at hand on-line on your own system without having to search for a misplaced or unintentionally discarded User manual or for an Internet connection. Besides allowing system simulations for measurement data generation and its subsequent processing, actual Application Data can also be processed by TK-MIP, by accessing sensor data via serial port,  via parallel port, or via a variety of commercially available Data Acquisition Cards (DAQ) using RS-232, PXI, VXI, GPIB, Ethernet protocols (as directed by the User) from TK-MIP’s MAIN MENU. TK-MIP is independent stand-alone software unrelated to MATLAB® or SIMULINK® and, as such, does NOT rely on or need these products and TK-MIP RUNS in only 16 Megabytes of RAM. Unfortunately, according to an October 2009 meeting at The MathWorks, their Data Acquisition Toolbox above currently lacks the ability to handle measurements using the older VME and PCI protocols, PCI, nor the ability to handle the recent PCIe protocol. In 2010, the Data Acquisition Toolbox does support PCIe protocol but still not VME [159]. TeK Associates believes in being backwards compatible not only in software but also in hardware and in I/O protocols. Since TK-MIP outputs its results in an ASCII formatted matrix data stream, these outputs may be passed on to MATLAB® or SIMULINK® after making simple standard accommodations. TeK Associates is committed to keeping TK-MIP affordable by holding the price at only $499 for a single User license (plus $7.00 for Shipping and Handling via regular mail and $15.00 via FedEx or UPS Blue). Please click on Summary & Pricing button at top of this screen for a free software demo download.

TK-MIP software provides screens that prompt the USER on how to configure their system for real-time Data Acquisition: 

An ability to perform certain TK-MIP computations for IMM is provided within TK-MIP but doing so in parallel within the framework of Grid Computing is NOT being pursued at this time for two reasons: (1) The a’ priori quantifiable CPU load of this Kalman filter-based variation is modest (as are the CPU burdens of Maximum Likelihood Least Squares-based estimation and LQG/LTR control algorithms as well). (2) Moreover, it has been revealed in 2005 that there is a serious security hole in the SSH (Secure Shell) used by almost all systems currently engaged in grid computing [37].

Multiple Models of Magill (MMM) and Interactive Multiple Models (IMM) consisting of Banks-of-Kalman-Filters 
depicted as processing in parallel but which may be implemented on mere Von Neumann sequential machines:

Von Neumann sequential machine (https://www.britannica.com/technology/von-Neumann-machine, https://www.geeksforgeeks.org/difference-between-von-neumann-and-harvard-architecture/)

An application example using IMM:

MMM architecture:

Flow charts representative of distinctive aspects of various alternative Multi-target tracking approaches:

IMM architecture:

An example application of Multi-target considerations: 

Please click here to view a more recent approach that utilizes Neural Networks to handle some important aspects of Multi-target Tracking.

While multi-target tracking in 3-dimentions using Dynamic Programming was too large a computational burden
to be practicable in the middle 1970's even with Robert E. Larson's algorithmic simplifications; he turned 
the problem around to be practical (by restricting it to 2-D) as an implementation of a collision-avoidance 
approach for cooperative surface ships (in 2-dimensions). Fifty years later, Aurora Flight Systems (now a 
Boeing company) is seeking to apply a Dynamic Programming approach to 3-dimensional cooperative collision 
avoidance (i.e., to an FAA cooperative multi-target tracking approach) along with MIT Lincoln Laboratory and
eventually others:
(https://www.prnewswire.com/news-releases/raytheon-lockheed-martin-sign-teaming-agreement-to-pursue-contract-to-modernize-surveillance-and-air-traffic-control-radar-systems-300821750.html) 
to implement and to perform initial testing.

TK-MIP offers pre-specified general structures, with options (already built-in and tested) that are merely to be User-selected at run time (as accessed through an easy-to-use logical and intuitive menu structure that exactly matches this application technology area). This structure expedites implementation by availing easy cross-checking via a copyrighted proprietary methodology [involving proposed software benchmarks of known closed-form solution, applicable to test any software that professes to solve these same types of problems] and by requiring less than a week to accomplish a full scale simulation and evaluation (versus the 6 weeks to 6 months usually needed to implement from scratch by $pecialists). USER still has to enter the matrix parameters that characterize or describe their application (an afternoon’s work if they already have this description available, as is frequently the case)
but is not required to do any programming per se when the application structure is exclusively linear and time-invariant. Please click this link: to see how to obtain the necessary models to be used. (To return to this point afterwards to resume reading, merely use the back arrow at the top left of your Web Browser or the keyboard: ALT + Left Arrow). TK-MIP outputs can also be User-directed after KF/EKF/Batch LMS processing for display within the low cost PC-based Mapptitude™ GIS software by Caliper, or within the more widely known MAPINFO®, or within DeLorme’s or ESRI’s mapping software.

Click this link to view the TK-MIP BANNER Screen (and some of its options). To return to this point afterwards to resume reading, merely use the back arrow at the top left of your Web Browser or the keyboard: ALT + Left Arrow.

Click this link to view the TK-MIP MAIN MENU screen (and some of its options). To return to this point afterwards to resume reading, merely use the back arrow at the top left of your Web Browser or the keyboard: ALT + Left Arrow.

Click this link to view the TK-MIP screen to MATCH S/W TO APP (and some of its alternative processing paths offered within TK-MIP that the USER is to click on to select proper match to their application's structure to that of TK-MIP internal processing without needing to perform any programming themselves). To return to this point afterwards to resume reading, merely use the back arrow at the top left of your Web Browser or the keyboard: ALT + Left Arrow. 

An earlier version of TeK Associates’ commercial software product, TK-MIP Version 1, was unveiled for the first time and initially demonstrated at our Booth 4304 at IEEE Electro ’95 in Boston, MA (21-23 June 1995). Our marketing techniques rely on maintaining a strong presence in the open technical literature by offering new results, new solutions, and new applications. Since the TK-MIP Version 2.0 approach allows Users to quickly perform Kalman Filtering\Smoothing\Simulation\Linear Quadratic Gaussian\Loop Transfer Recovery (LQG\LTR) Optimal Feedback Regulator Control, there is no longer a need for the User to explicitly program these activities themselves (thus avoiding any encounters with unpleasant software bugs inadvertently introduced) and USER may, instead, focus more on the particular application at hand (and its associated underlying design of experiment). This TK-MIP software has been validated to also correctly handle time-varying linear systems, as routinely arise in linearizing general nonlinear systems occurring in realistic applications. An impressive array of auxiliary supporting functions are also included within this software such as spreadsheet inputting and editing of system description matrices; User-selectable color display plotting on screen and from a printer with a capability to simply specify the detailed format of output graphs individually and in arrays in order to convey a story through useful juxta-positioned comparisons; and offering alternative tabular display of outputs; along with pre-formatted printing of results for ease-of-use and clarity by pre-engineered design; automatic conversion from continuous-time state variable or auto-regressive (AR) or auto-regressive moving average (ARMA) mathematical model system representation to discrete-time state variable representation [95] (i.e., as a Linear System Realization); facilities for simulating vector colored noise of specified character rather than just conventional white noise (by providing the capability to perform Matrix Spectral Factorization (MSF) to obtain the requisite preliminary shaping filter). [These last two features are only found in TK-MIP to date. There is more on MSF to follow next below.]  

Another advantage possessed by TK-MIP® over any of our competitor’s software is that we provide the only software that successfully implements Matrix Spectral Factorization (MSF) as a precedent. MSF is a Kalman Filtering accoutrement that enables the rigorous routine handling (or system modeling) of serially time-correlated measurement noises and process noises that would otherwise be too general and more challenging than could normally be handled within a standard Kalman filter framework that expects only additive Gaussian White Noise (GWN) as input. Such unruly, more general noises are accommodated within TK-MIP via a two-step procedure of (1) using MSF to decompose them into an associated dynamical system Multi-Input Multi-Output (MIMO) transfer function ultimately stimulated by (WGN) and then (2) our explicitly implementing an algorithm for specifying a corresponding linear system realization representing the particular specified MIMO time-correlation matrix or, equivalently, its power spectral matrix. Such systems structurally decomposed in this way can still be handled or processed now within a standard estimation theory framework by just increasing the original systems dimension by augmenting these noise dynamics into the known dynamics of the original system, and then both can be handled within a standard state-variable or descriptor system formulation of somewhat higher dimensions. These techniques are discussed and demonstrated in:

A more realistically detailed model may be used for the system simulation while alternative reduced-order models can be used for estimation and\or control, as usually occurs in practice because of constraints on tolerable computational delay and computational capacity of supporting processing resources, which usually restricts the fidelity of the model to be used in practical implementations to principal components that hopefully “capture the essence” of the true system behavior. Simulated noise in TK-MIP can be Gaussian white noise, Poisson white noise (but not usually and almost never), or a weighted mixture of the two (with specified variances being provided for both as User-specified inputs) as a worse case consideration in a sensitivity evaluation of application designs. Filtering and\or control performance sensitivities may also be revealed under variations in underlying statistics, initial conditions, and variations in model structure, in model order, or in its specific parameter values. Prior to pursuing the above described detailed activities, Observability\Controllability testing can be performed automatically (for linear time-invariant applications) to assure that structural conditions are suitable for performing Kalman filtering, smoothing, and optimal control. A novel aspect is that there is a full on-line tutorial on how to use the software as well as describing the theoretical underpinnings, along with block diagrams and explanatory equations since TK-MIP is also designed for the novice student Users from various disciplines as well as for experts in electrical or mechanical engineering, where these techniques originated. The pedagogical detail may be turned off (and is automatically unloaded from RAM during actual signal processing so that it doesn’t contribute to any unnecessary overhead) but may be turned back on any time a gentle reminder is again sought. A sophisticated proprietary pedagogical technique is used that is much more responsive to immediate USER questions than would otherwise be availed by merely interrogating a standard WINDOWS 9X/2000/ME/NT/XP Help system and this is enhanced by exhibiting descriptive equations when appropriate for knowledgeable Users. TK-MIP also includes the option of activating a modern version of square root Kalman filtering (for effectively achieving double precision accuracy without actually invoking double precision calculations nor incurring additional time delay overhead beyond what is present for a standard version of Kalman filtering) which is a numerically stable version for real-time on-line use, that provides a software architecture for benignly handling round-off, where this type of robustness is important over the long haul of continuous real-time operation. The full history pertaining to the development of squareroot filters (over 15 or 20 prior years by others) originally required many more operations than a standard Kalman Filter and therefore incurred more delay in associated processing. Implementation processing delay for both Carlson and Bierman U-D-U^T square root filters is "within the same ballpark" of what is incurred for a standard Kalman filter but has twice the precision and is "numerically stable" or "computationally stable" as well (an extremely desirable feature). Moreover, the Bierman U-D-U^T version propagates U and D forwards in time and only puts the pieces together when calculating the covariance of estimation error P for outputting as P = U∙D∙U^T. Dr. Eli Brookner's (Raytheon-retired) book also distinguishes between ordinary Kalman filtering and squareroot filtering using the electrical analogy of, respectively, the "power" form or the "voltage" form. Where the term "voltage form" refers to both Bierman's U-D-U^T squareroot or Carlson's squareroot versions (also see [153]).

TK-MIP provides clear insightful explainations of underlying theorectical aspects involving both implementation and statistical 
considerations and lucrative alternative approaches that can be followed to implement (approximate) Nonlinear Filtering:

Analytic derivation of the Fokker-Planck Partial Differential Equation (PDE) for the time evolution of the probability density function for a nonlinear Ito stochastic state-variable system model with process noise being a vector Weiner process [which looks like an Ordinary Differential Equation (ODE)]

Additional discussion of the class of Alpha-stable filters mentioned above:
http://math.bu.edu/people/mveillet/html/alphastablepub.html#:~:text=The%20alpha-stable%20distribution%20is%20a%20four-parameter%20family%20of,distribution%20is%20right-%20%28%29%20or%20left-%20%28%29%20skewed. 
https://www.computer.org/csdl/proceedings-article/icassp/2000/00861844/12OmNAGw17e, https://dl.acm.org/doi/abs/10.1109/78.875458, https://dl.acm.org/doi/10.1109/78.622952 

Simulation Demos at earlier IEEE Electro: Although the TK-MIP software is general purpose, TeK Associates originally demonstrated this software in Boston, MA on 21-23 June 1995 for (1) the situation of an aircraft equipped with an Inertial Navigation System (INS) using a Global Positioning System (GPS) receiver for periodic NAV updates;(2) several benchmark test problems of known closed-form solution (for comparison purposes in verification\validation). These are both now included within the package as standard examples to help familiarize new Users with the workings of TK-MIP.

This TK-MIP software program utilizes an (n x 1) discrete-time Linear SYSTEM dynamics state variable model of the following form:

(Eq. 1.1)                       x(k+1) = A x(k) + F w(k) + [ B u(k) ],

with

                                      x(0) = x₀ (the initial condition)

and an (m x 1) discrete-time linear Sensor MEASUREMENT (data) observation model of the following algebraic form:

(Eq. 1.2)                       z(k) = C x(k) + G v(k),

where:

                          x(k) is the System State vector at time-step "k",                                   (TK-MIP Requirement is that this is to be USER supplied)

                         A is the (n x n) System Transition Matrix,                                            (TK-MIP Requirement is that this is to be USER supplied)

                          z(k) is the (m x 1) observed sensor data measurement 

                                 vector at time-step "k",                                                                    (TK-MIP Requirement is that this is to be USER supplied)

                          C is the (m x n) Observation Matrix,                                                     (TK-MIP Requirement is that this is to be USER supplied)

                                    w(k) & v(k) are INDEPENDENT Gaussian White Noise (GWN) vector sequences representing process and measurement noises with
                                    specified variances: Q & R, respectively.

                          F is the process noise Gain Matrix,                                                       (TK-MIP Requirement is that this is to be USER supplied)

                          G is the measurement noise Gain Matrix,                                             (TK-MIP Requirement is that this is to be USER supplied)

                          u is an (optional) exogenous control input sequence,                         (TK-MIP Requirement is that this is to be USER supplied, if control present in application)

                          B is the (optional) Control Gain Matrix for the above control.               (TK-MIP Requirement is that this is to be USER supplied, if control present in application)

[The above matrices A, C, F, G, B, Q, R can be time-varying explicit functions of the time index "k" or be merely constant matrices. And for the nonlinear case considered further below, these matrices may also be a function of the states (that are later eventually replaced in the FILTER Model by "one time-step behind estimates" when an EKF or a 2^nd order EKF is being invoked).] A control, u, in the above System Model may also be used to implement "fix/resets" for navigation applications involving an INS by subtracting off the recent estimate in order to "zero" the corresponding state in the actual physical system at the exact time when the control is applied so that the estimate in both the system and Filter model are both simultaneously zeroed (this is applied to only certain states of importance and not to ALL states). See [95] for further clarification.

In the above discussion, we have NOT yet pinned down or fixed the dimensions of vector processes w(k), v(k), NOR the dimensions of their respective Gain Matrices  

F, and G here. I am leaving these open for NOW except to say that they will be selected so that the whole of the SYSTEM dynamics equation and algebraic SENSOR 

data measurements are both properly "conformable" where they occur in matrix addition and matrix multiplication.

The open matrix and vector dimensions will be explicitly pinned down in connection with a further discussion of symmetric Q being allowed to be merely positive 

semi-definite and eventually have a smaller fundamental core (of the lowest possible dimension) that is strictly positive definite for a minimum dimensioned process 

noise vector. (So further analysis will clear things up and pin down the dimensions to their proper fixed values NEXT!)

See or click on: https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Degenerate_case  [Especially see Degenerate Case of section just prior.]
From Probability and Statistics (topics that we had looked up more than 40 years ago), it is fairly well-known for Gaussians, that if the corresponding covariance 
matrix is only positive semi-definite, then the random variables being considered are linear (more specifically, affine) combinations of the total underlying Gaussians, 
the number of independent Gaussians present corresponds to the rank of the covariance matrix. The problem can be reduced or simplified to be a consideration of 
only those fundamental underlying Gaussian variates that are "independent", and their number corresponds to the rank of the original larger Covariance Matrix.
The same is true of vector Gaussian radom processes. Likewise, the same is true of the covariances of vector Gaussian white noises.

[For the cases of implementing an Extended Kalman Filter (EKF), or an Iterated EKF, or a Gaussian Second Order Filter (a higher order variant of a EKF that utilizes the first three terms within the multidimensional Taylor Series Expansion {including the 1^st derivative with respect to a vector, being the Jacobian, and the the 2^nd derivative with respect to a vector, being the Hessian}, as obtained outside of TK-MIP, perhaps by hand-calculation) that is being used as a close local approximation to the nonlinear function present on the Right Hand Side (RHS) of the following Ordinary Differential Equation (ODE) representing the system as: 
(Eq. 2.1)  dx/dt = a(x,u,t) + f(x,u,t)w(t) + [ b(x,t)u(t) ],                                                                    (TK-MIP Requirement is that this is to be USER supplied)
(Eq. 2.2)     z(t) = c(x,u,t) + g(x,u,t)v(t)                                                                                          (TK-MIP Requirement is that this is to be USER supplied) 
After the above nonlinear system has been properly linearized by the USER, the resulting matrices A, C, F, G, B, Q, R can be time-varying explicit functions of the time index "k" AND the state "x" AND control "u" (if present) or be merely constant matrices. TK-MIP can also be used to computationally handle this more challenging and more general nonlinear case but estimation results are merely "approximate" (but, perhaps, "good enough") but not "optimal" since the estimates provided via this route are no longer the "conditional expectation of the states, given the measurements" (as required for an "optimal estimator", which, in general, would be intractable in real-time as the important reason or motivation for this necessary trade-off).]  The trade-off incurred between Optimal nonlinear estimation (unobtainable because it is intractably complex within the time allotted between newly available sensor measurements to be processed) and tractable approximate nonlinear estimation are discussed next to offer some insights into what is involved in a reasonable approximation and what needs to be done. 

For Linear Kalman Filters for exclusively linear system models and independent Gaussian Process and Measurement noises, it is fairly straight-forward to handle this situation with only discrete-time filter models, as already addressed above. For similarly handling approximate nonlinear estimation with either an Extended Kalman Filter (EKF) or a Gaussian 2^nd Order Filter, there are three additional steps that must be performed (that we also provide access to the USER within TK-MIP). (1) Step One: A Runge-Kutta (RK) 4(5) or, preferably, a Runge-Kutta-Fehlberg 4(5) method, with automatically adaptive step-size https://maths.cnam.fr/IMG/pdf/RungeKuttaFehlbergProof.pdf integration of the original nonlinear ODE must be performed between measurements (as a " continuous-time system" with " discrete-time measurement samples" available, as explained in [95]); (2) Step Two: this R-K needs to be applied for the approximate estimator (KF) and for the entire original system [as needed for system to estimator cross-comparisons in determining how well the linear approximate estimator is following the nonlinear state variable "truth model": (3) Step Three: the USER must return to the Database (in defining_model), where the Filter model (KF) was originally entered after the required linearization step had been performed and the results entered. Now everywhere there is a state (e.g., x₁) in the database for the Filter Model, it needs to be replaced by the corresponding estimator result from the previous measurement update step (e.g., xhat₁, respectively). This replacement needs to be performed and completed for every state that appears in the Filter model in implementing either an Extended Kalman Filter (EKF) or in implementing a Gaussian 2nd Order filter. Examples of properly handling these three aspects are offered in: .Kerr, T. H., “Streamlining Measurement Iteration for EKF Target Tracking,” IEEE Transactions on Aerospace and Electronic Systems, Vol. 27, No. 2, pp. 408-421, Mar. 1991 and in http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDoc.pdf&AD=ADP011192. 

Insights into the trade-off incurred between magnitude of Q versus magnitude of R as it relates, in simplified form, to the speed of convergence of a Kalman filter is offered for a simple scalar numerical example in Sec. 7.1 of Gelb [95]. Consider that Q not being positive definite for the Matrix case is tantamount to q being zero for the scalar case; so let's consider the limiting case as q converges to zero. In particular, Example 7.1-3 for estimating a scalar random walk from a scalar noisy measurement, where the process noise Covariance Intensity Matrix, Q, for this scalar case is "q" and where the measurement noise Covariance Intensity Matrix, R, for this scalar case is "r"; then the resulting steady-state covariance of estimation error is sqrt(rq) and the resulting steady-state Kalman gain is sqrt(q/r). Going further to investigate the behavior of the limiting answer as both q and r become vanishingly small, take q = [q'/j²] and take r = [r'/j²], then the resulting steady-state covariance of estimation error is Lim _j→∞ {[sqrt(rq)]/j²} = 0 (since the j²'s all divide out) and the resulting steady-state Kalman gain is finite as: Lim _j→∞ {sqrt(q/r)} = sqrt(q'/r') (a finite constant). Another numerical example offered as Sec. 7.1-4 is three different design values to be cxonsidered for Kalman filter convergence as a function of q, r, and the time constant of a first order Markov process. These are general principles of Kalman filter convergence behavior as a function of these noise covariances that have been known for 5 decades. A clearer exhibition of the effect of "Q" and "R" (or scalar "q" and scalar "r", respectively) on the convergence of a simplified 2-state Kalman filter (for illustrative purposes) is conveyed in:
Bertil Ekstrand, "Analytical Steady-State Solution for a Kalman Tracking Filter," IEEE Trrans. on Aerspace and Electronic Systems, Vol. 19, No. 6, pp. 815-819, Nov. 1983.

The white noise w(.):

·         is of zero mean: E [ w(k) ] = 0 for all time steps k,

·         is INDEPENDENT (uncorrelated) as

            E [ w(k) w^T(j) ] = 0 for all k not equal to j (i.e., k ≠  j),

      and as

          E [ w(k) w^T(k) ] = Q for all k,                                                                                                     (TK-MIP Requirement is that this is to be USER supplied)

      where 

           Q = Q^T > 0,            (TK-MIP Requirement is that USER has already verified this property for their application so that estimation within TK-MIP will be well-posed)

      (i.e., Q = Q^T > 0  above, is standard short notation for Q being a symmetric and positive definite matrix. Please see [71] to [73] and [92] where we, now at TeK Associates, in performing IV&V, historically corrected (all found while under R&D or IV&V contracts during the 1970's and early to mid-1980's) several prevalent problems that existed in the software developed by other organizations for performing numerical verification of matrix positive semi-definiteness (and positive definiteness) in many important DoD applications). Also see http://www2.econ.iastate.edu/classes/econ501/Hallam/documents/Quad_Forms_000.pdf, https://onlinelibrary.wiley.com/doi/pdf/10.1002/9780470173862.app3. Apparently, no better numerical tests are offered within these two more recent and definitive alternate characterizations of positive definiteness and positive semi-definiteness yet, thankfully, others are still trying: see New insights into matrix semi-definiteness: https://mjo.osborne.economics.utoronto.ca/index.php/tutorial/index/1/qfs/t.

·         is Gaussianly distributed [denoted by w(k) ~ N(0,Q) ] at each time-step "k",

·         is INDEPENDENT of the Gaussian initial condition x(0) as: 

            E [ x(0) w^T(k) ] = 0 for all k,

      and is also nominally INDEPENDENT of the Gaussian white measurement noise v(k):

          E [ w(k) v^T(j) ] = 0 for all k not equal to j (i.e., k ≠  j),

      and likewise for the definition of the zero mean measurement noise v(k) 

      except that its variance is R,                                                                                            (TK-MIP Requirement is that this is to be USER supplied)

      where in the above  

              w^T(·)

      represents the transpose of w(·) and the symbol E [ · ] denotes the standard unconditional expectation operator. 

For estimator initial conditions (i.c.'s), it is assumed that it is Gaussianly distributed as N( x_o, P_o) so that

the initial estimate at time = t_o is:

xhat(t_o)  = x_o,                                                                                                                           (TK-MIP Requirement is that this is to be USER supplied)

the initial Covariance at time = t_o is:

P(t_o)  = P_o.                                                                                                                               (TK-MIP Requirement is that this is to be USER supplied)

In the above, the USER (or analyst) usually does not know what true value of x_o to use to get the estimation algorithm started and, similarly, what value of P_o to be used to start the estimation algorithm. The good news is that if the original system and measurement model are “controllable” in the process noise and “observable” in structure, then for the linear case, the Kalman filter is analytically provably to be “exponentially and asymptotically stable” and quickly converges to the correct answer at an asymptotically exponential rate even if it is started off wrong with an incorrect value for  x_o and an incorrect value for P_o, as long as the starting P_o is "positive definite" (easily achieved if it is taken to be diagonal with all entries positive). Even if the original linear system and measurement model are not controllable and observable, respectively, it will still converge to the correct values reasonably quickly but not as fast as it would if they were controllable and observable. For situations where the original system is NOT “controllable AND observable”, the Kalman filter can still work well but will generally take longer to converge, as a slight inconvenience. The closer the initial guesses or starting values are to the actual but unknown values, the better or faster it will converge to the "right answers" (since the initial guesses are less far away and closer to the ideal goal).  

For applications involving very accurate Inertial Navigation Systems (INS), as for U.S. SSBN's and SSN's for example, it is usually the case that one only uses the exact values that have been determined for Q (after laborious calibration on a test stand, perhaps by others). Values used for Q in these types of INS applications are like accounting problems that seek exactness in the values used. These values are assumed to have already passed some earlier off-line positive definiteness test or else they would have never gotten so far as to be documented for subsequent data entry for this type of application.

Admittedly, Q-tuning is more prevalent in target tracking applications. For a time-varying Q(k), it may ideally need to be checked for positive semi-definiteness/positive definiteness at each time step. Sometimes such frequent cross-checking is not practicable. An alternative to continuous checking for positive semi-definiteness at each time-step is to provide a "fictitious noise", also known as "Q-tuning", to be positive definite, according to the techniques offered in [82] and [83]. There is also a simple alternative to "Q-tuning" for both the cases of constant and time-varying Q: just by numerically nudging the effective Q to be slightly more "diagonally dominant" as Q_{{modified once}}(k)= Q_{original}(k)+ ß· diag[Q_{original}(k)], where diag[Q_{original}(k)] is a diagonal matrix consisting only of the "main diagonal" (i.e., top left element to bottom right element) of Q_{original}(k) and all diagonal terms must be positive and the scalar, ß, is a USER specified fixed constant 0 ≤ ß ≤ 1. The theoretical justification for this particular approach to "Q-tuning" is provided by/obtained from Gershgorin disks: https://en.wikipedia.org/wiki/Gershgorin_circle_theorem. "Q-tuning" is, in fact, an Art rather than a Science despite [82] and [83] that attempt to elevate it to the status of a science. Despite what we said earlier above about INS applications usually requiring exact cost accounting, the application for which the "Q-tuning" methodology was developed and applied in [82] involved an airborne INS utilizing GPS within an EKF. Contradictions such as this abound! Implementers will do anything (within reason) to make it work (as well they should)! In particular, notice that when the USER makes the scalar ß = 0, then the original Q in the above is unchanged!

Emeritus Prof. Yaakov Bar-Shalom (UCONN), who worked in industry at Systems Control Incorporated (SCI) in Palo Alto, CA for many years before joining academia, has many wonderful stories about "Q-tuning" a Kalman tracking filter: in particular, he mentions one application where the pertinent Q-tuning was very intuitive but the resulting performance of the Kalman filter was very bad or disappointing and another application, where the Q-tuning that he used was counter-intuitive yet the Kalman filter performance was very good. Proper Q-tuning  is indeed an art.

Contrasted to the situation for Controllability involving "a controllable pair" (A,B) where all states can be influences by the control effort exerted (which is a good property to possess when seeking to implement a control), while possessing Controllability involving "a controllable pair" (A,F), where all states can be influenced and adversely aggravated by the process noise present (is a bad characteristic to possess). When the underlying systems are linear and time-invariant, the computational numerical tests to verify the above two situations are: rank[B|A·B|A²·B|...|A^(n-1)·B] = n and rank[F|A·F|A²·F|...|A^(n-1)·F] = n, respectively. The augmented matrices that were checked to see if they had rank = n (the dimension of the state) are Called the Controllability Grammian. A corresponding augmented matrix involves transposes throughout and is called the "Observability Grammian". “Observability” and "Controllability” yea/nay tests for linear systems with time-varying “System matrix”, “Obsevation matrix”, and “System Noise Gain matrix” are presented in [59], as developed by Leonard Silverman, but are difficult to implement for a general time-varying case; so it is not used within TK-MIP. If desired, a USER may perform their own specialized investigation using this controllability test to satisfy their own suspicions regarding a proper answer. Over a short enough time-step, system structure and parameter values are essentially CONSTANT. For discrete-time sample value systems, answers to these types of questions are available within "n" time-steps, where "n" is the dimension or state-size of the system being investigated.

Returning to the model, already discussed in detail above (but repeated here again for convenience and for further more detailed analysis), that TK-MIP utilizes as an (n x 1) discrete-time Linear SYSTEM dynamics state variable model of the following form:

(Eq. 3.1)                        x(k+1) = A x(k) + F w(k) + [ B u(k) ], where the process noise w(k) is WGN ~ N(0_n, Q)

with

                                           x(0) = x₀ (the initial condition), where x₀ is from a Gaussian distribution N(x_o, P_o)

and an (m x 1) discrete-time linear Sensor MEASUREMENT (data) observation model of the following algebraic form:

(Eq. 3.2)                            z(k) = C x(k) + G v(k), where the measurement noise v(k) is WGN ~ N(0_m, R);

then, according to Thomas Kailath (emeritus Prof. at Stanford Univ.) that, without any loss of generality, the above model, described in detail earlier above, is equivalent to:

(Eq. 4.1)                        x(k+1) = A x(k) + I_{(n x n)} w(k) + [ B u(k) ],  with identity matrix I_{(n x n)} and process noise w(k) distributed as N(0_n, F·Q·F^T); notation: w(k) ~ N(0_n, F·Q·F^T)

with

                                           x(0) = x₀ (the initial condition), where x₀ is from a Gaussian distribution N(x_o, P_o)

and an (m x 1) discrete-time linear Sensor MEASUREMENT (data) observation model is again of the following algebraic form:

 (Eq. 4.2)                            z(k) = C x(k) + G v(k), where the measurement noise v(k) is WGN ~ N(0_m, R).

The distinction between the above two model representations is only in the system description, specifically in the Process Noise Gain Matrix, now an Identity matrix, and the associated covariance of the process noise, now having the covariance F·Q·F^T. Such a claim is justified since both representations have the same identical Fokker-Planck equation in common (please see Appendix 4 in [124] or last three pages of [94] explicitly available from this screen below) and consequently, they have the exact same associated Kalman filter when implemented (except for possible minor "tweeks" that can occur in software within possible software author personalization).

Reconsidering the above previously mentioned Measurement Noise Controllability Test: rank[F|A·F|A²·F|...|A^(n-1)·F] = n?, based on Prof. Thomas Kailath's argument above that, without any loss of generality, one can instead focus on F·Q·F^T rather than on Q and F separately in the system dynamics model's description and further factor it into two Cholesky factors, CHOLES, as 

F·Q·F^T = CHOLES·CHOLES^T. Now a more germane test for noise controllability, involving both pertinent parameters of F and Q simultaneously, which now involves checking whether rank[CHOLES|A·CHOLES|A²·CHOLES|...|A^(n-1)·CHOLES] = n?

While possessing a Cholesky (http://www.math.utoledo.edu/~codenth/Linear_Algebra/Calculators/Cholesky_factorization.html) does serve as a valid test of Matrix positive definiteness and indicates problems being present when it breaks down or fails to go to successful completion when testing a square symmetric matrix for positive definiteness, a drawback is that the number of operations associated with its use is O(n³). There is a version or variation on a direct application of a SVD (already declared by others as the best method for determining or revealing a matrix's definiteness, semi-definiteness, or lack thereof) known as "Aarsen's method" [92] that exploits the symmetry of the matrix under test to an advantage and is only O(n³), but lower at n³/6, in the number of operations required (for testing Q and P ) and O(m³) for testing R. By way of comparison, the QR Algorithm for symmetric matrices requires O(n²) operations per step [109, p. 414] for n steps so still O(n³) in  total. 

While having a non-positive definite Q-matrix may seem to be a boon by indicating that the process noise does not corrupt all the states of the system in its state variable representation and, similarly, having a non-positive definite R-matrix may be interpreted as being a boon by not all of the measurements being tainted by measurement noise, there can be computational reasons why full rank Q and R are desirable in order to avoid computational difficulties, at least for the standard Kalman filter for the linear case. For the case of long run times, only the so-called Squareroot Kalman Filter version is "numerically stable" and can tolerate lower rank Q and low rank R without encountering problems with numerical sensitivity.

SAVING THE BEST FOR LAST: Frequently, "Q-tuning" is invoked when using a reduced-order filter for an application that has a much larger dimensioned "Truth model". The underlying idea being that one attempts to capture the essence of the application's behavior (as in an approach to identifying the "Principal Components", which describe the essential behavior of the underlying system) with the reduced-order (lower-order) filter model utilized. Then one makes use of fictitious "Q-tuning" to approximately account for the "unmodeled dynamics" NOT explicitly included in the reduced-order "filter model". Sometimes, the approximate model resorted to is essentially a WAG (i.e., an outright "Wild Ass Guess", please excuse my use of this well-known expression in this context). Surprisingly, this frame work just described can be somewhat forgiving as long as it is guided by good engineering insight and a diligent attempt to capture the pertinent (and prominent) system behavior and characteristics. A numerical example to put various dimensions for a particular reduced-order Kalman filter in perspective follows next. For U.S. Poseidon SSBN's some fifty years ago [106], Sperry Systems Management (SSM) in Great Neck, NY had established a detailed error model for the G-7B INS, a conventional 3-input axis spinning rotor gyro: it's detailed error (Truth) model consisted of 34 states. It's associated Kalman Filter model consisted of only 7 states and was called the 7-State STAtistical Reset Filter (STAR filter). Also see page 282 of [115]. (The identities of these states are not divulged in either references [106] or [115].) Thanks to the empirical Moore's Law since 1965 and into the near future, medium future, and far future, trends in available computer resources to support implementation of Kalman filters and their respective system and measurement models are considerably less constraining and consequently now tolerate more detailed larger dimensional models conveying a more accurate representation of the application system and measurement structure.

The importance or impact of symmetric matrices being positive definite versus being merely positive semi-definite can best be understood by considering their impact on the corresponding "quadratic forms" involving these symmetric matrices. For y = x^TQx, taking Rⁿ into R¹, the shape of the paraboloid is "strictly convex" and y is positive for all x ≠  0, while for a matrix Q that is positive semi-definite, there is no longer guaranteed convexity and there can be some flatness in the associated paraboloid when y is used as a "cost function". Having convex cost functions is useful when attempting to intersect regions of interest so that the results will be well-behaved and not pathological. Such issues arise in minimum energy Linear Quadratic Optimal Control regulators for both Finite Planning Horizons (integrating the two term integrand:∫_o^τ [x^T(t)Q(t)x(t) + u^T(t)R(t)u(t)] dt (with respect to time) from 0 to finite positive τ) and for Infinite Planning Horizons (integrating the two term integrand: ∫_o^∞ [x^T(t)Qx(t) + u^T(t)Ru(t)]dt (with respect to time, from 0 to infinity: ∞). The correspondingly appropriate existing constraint is that the system dynamics is described by a linear Ordinary Differential Equation (ODE) of the standard form dx/dt = Ax(t) + Bu(t), where the control, u(t), is to be determined to minimize the two term cost functions previously presented and to have the feedback form u(t) = GAIN·[x(t) - z(t)] = GAIN· [I_{n x n} - C] x(t). Indeed, there is a long recognized "duality" between the Optimal Linear Quadratic Regulator problem and the Linear Kalman Optimal Filter problem (see Blackmore, P. A., Bitmead, R. R., “Duality Between the Discrete-Time Kalman Filter and LQ Control Law,” IEEE Transactions on Automatic Control, Vol. AC-40, No. 8, pp. 1442-1443, Aug. 1995). Also see [70], where we discuss even more Kalman Filter applications): both involve calculating the proper gain matrix to use and both involve solving a Riccati equation to obtain the proper gain matrix. A Riccati equation is to be solved forwards in time for the Kalman Filter while another Riccati equation is to be solved backwards in time for the minimum energy Linear Quadratic Regulator. Other applications where convex functions over convex regions are beneficial in further analysis in order that the results be analytically well-behaved are provided by us in [110] - [116] (please see [117] - [123], [127] for further independent confirmation/substantiation by others of the usefulness of convex functions and convex sets in this way within the field of engineering and optimization). For both the "Finite Planning Horizon" and the "Infinite Planning Horizon", the R and Q appearing within the corresponding integral cost functions, respectively, are still symmetric and positive definite but are NOT covariance intensity matrices, as defined following Eqs. 1.1, 1.2, 2.1, 2.2, 3.1, 3.2, 4.1,4.2 but play a similar role in the corresponding associated Riccati Equations, since the linear "quadratic regulator" control problem is the mathematical "dual" of the linear filtering problem, as originally recognized by Rudolf Kalman back in the 1960's when he correctly posed and solved both using similar techniques with similar resulting solution structure. 

The image below portrays the delineation between the structural situations for those applications that warrant use of a Luenberger Observer (LO) and those that warrant use of a Kalman Filter (KF) and the "regularity conditions" that support its use for applications of state estimation and/or additionally Linear Quadratic (LQ) Minimum Energy Control. Easily accessible explanations of LO are available in [9], [99], [100].

These prior definitions above are repeated again within the section on "Defining Model(s)" as a convenient refresher for the USER closer to the discussion where it is to be performed by the USER as "actionable" in modern business parlance. The following more concise discussion is from actual screen shots of TK-MIP: which also defines the pertinent vector and matrix dimensions:

      Notice in the above screen shot that possible correlation between the process noise and the measurement noise is acknowledged. When that occurs, instead of the Kalman Gain being: 

Eq. 4.3   K_k = P_k|k-1C_k^T [C_kP_k|k-1C_k^T + R_k]^-1, 

    it then becomes 

Eq. 4.4  K_k = P_k|k-1C_k^T [C_kP_k|k-1C_k^T + S_k + R_k]^-1 [107], 

     where, in the above, matrix S_k is the conformable correlation Matrix between GWN process noise and GWN measurement noise (denoted by [ QR ] in the above image). While there is an algorithm known as the QR algorithm, the above is NOT referencing the QR algorithm but is merely notation for the pertinent cross-correlation that may exist in some applications between process noise and measurement noise. Apparently, the previous simple summarizing expression is only possible when the dimension of the system (or plant) noise is identical to the dimension of the measurement or sensor noise (not a very realistic situation in most applications). Examples of practical engineering situations where such structures may be useful are: (1) for INS-stabilized missiles, drones, and UAVs that may use occasional or periodic external position fixes, wind buffeting may affect INS System model noise intensity that may then directly adversely affect lever arm positioning of external positioning sensor (with respect to their center-of-gravity) as it makes use of the external measurements to compensate for INS build-up of gyro-drift; (2) on a U-2 (or, perhaps, by now, on a U-3) that seeks to use INS-pointing to take simultaneous multi-sensor images in order to triangulate or get a visual position fix enroot. Upon inserting the above modified sequence for the Kalman Gain K_k that includes S_k within the well-known Joseph Form for the Covariance of Estimation Error update equation, according to the tenets of standard Covariance Analysis when determining an Error Budget, one obtains the correct Covariance of Error associated with this now modified Kalman Gain matrix that includes the cross-correlation S_k between System and Measurement noises. Also see: Y. Sunahara, A. Ohsumi, K. Terashima, H. Akashi, "Representation of Solutions and Stability of Linear Differential Equations with Random Coefficients via Lie Algebraic Theory," 11th JAACE Symposium on Stochastic Systems, Kyoto, pp. 45-48, 27-29 Nov. 1979.

      My earliest extensive Kalman filter modeling was for a ship-borne Inertial Navigation System (INS) that was aided by using a variety of alternative "navaids" for
position fixes (but such fixes did not strictly occur periodically but on an "as available and sought for use" basis as "fixes of opportunity"). Such an INS error model, 
allows the Kalman filter to capture or properly emulate the effect of a "fundamental underlying and growing (as time elapses) unbounded gyro drift-rate" being present. 
However, it is properly compensated for or the effect is ameliorated by using external navaid fixes (e.g., LORAN-C, VOR/ME, GPS, etc.) to correct the on-board INS 
in real-time using the effect of these measurement fixes as simple control actions performed on the outputs of the actual INS hardware as corrective "fix
resets". The INS error model, while linear, has elements in its system matrix and, correspondingly, in the "system matrix" of its representative Kalman filter model that 
are sinusoidal functions (i. e., sines and cosines) of Latitude, Longitude, and time. The actual output readings of the INS (with its fundamentally nonlinear hardware 
mechanization) are corrected in real-time by the outputed Kalman filter estimates of the corresponding error states that are immediately physically subtracted in
analog fashion from the whole value outputs of the INS hardware. (When they are subtracted, the corresponding states in the INS Kalman filter model should also be
simultaneously "zeroed out" as well to properly correspond). Similarly, the values of the corresponding covariances should be changed to their initial values
(as originally used after original calibration and thermal stabilization) but NOT zeroed. They should still be maintained as being effectively "positive definite" square
matrices, where a diagonal matrix with all entries positive suffices.

      Now, simulation studies can be performed to cross-compare resulting INS accuracies in a variety of application scenarios as a function of time and "navaid" usage at 
different indicated "measurement fix" times. In certain simulation situations, using a representative Nominal fixed value for associated Latitude and Longitude
suffices for sufficiently short mission time epochs as a satisfactory simplified approach. In other situations that seek greater fidelity in the simulation or in processing 
actual "real system data", this simplified "constant LAT and LONG approach" is not realistic enough and time-varying Latitude and Longitude should be properly 
accounted for and included within the processing considerations. If the application were for an airborne platform or for a submarine, altitude (or depth), respectively,
should also be included. One can accomplish this additional realism by using a separate position profile that includes the time variation of Latitude, Longitude,  
and Altitude/Depth as an input parameter file containing this information to be fed to the simulation or implementation of the Kalman filter (i.e., Navigation Filter) as 
an easy additional input. For handling this situation more realistically within TK-MIP, a trajectory file should be included and TK-MIP can accommodate this. A
simple trajectory file can be constructed by utilizing the nominal velocity (magnitude and direction) and properly incremented as a function of time as Latitude (in degrees, 
including corresponding minutes, converted to degrees and seconds converted to degrees and similarly for Longitude). As with its standard use in Calculus, all 
simulation calculations are usually computed in radians for scientific and engineering work relating to the solution of differential equations or corresponding difference
equations (sometimes referred to as the "iteration equations"), as a very familiar precedent.

      For radar target tracking applications, what is modeled within the Kalman Filter is the dynamics behavior of the designated target. In seeking to intercept an enemy 
Reenty Vehicle (RV) traveling in a ballistic trajectory, a proper model includes consideration of the effects of the inverse squared gravity, including the significant second
harmonic J₂ term to account for the oblateness of the earth, which is a spheroid and not a perfect sphere. Since the target's trajectory is ballistic during mid-course, using
merely the target's position and corresponding velocity at any point within its ballistic (i.e., "unpowered") trajectory at any point above the atmospheric drag needs to be used
within the tracking filter (and by so doing avoids needing to explicitly being informed of or needing to handle, discuss, or expose classified information related to the thrust
profile of the rockets or boosters needed to get the RV there. Knowledge of the radar cross-section of the target is needed as well as the explicit location of the tracking
radar (which ultimately enters into the measurement model observation matrix). Please click: For a nice clear discussion of Tracking Small Low-Earth-Orbit (LEO) Objects using a fence Type Radar.

     In pursuing radar applications, it is advisable to first get a nice overview by reading the following book:
    J.C. Toomay, RADAR PRINCIOLES FOR THE NON-SPECIALIST, LIFETIME LEARNING PUBLICATIONS, Bellmont, CA, A Division of Wadsworth, London, Singapore, Sydney, 
    Toranto, Mexico City, 1982.
     https://books.google.com/books?id=pd71EFWpVZMC&printsec=frontcover#v=onepage&q&f=false 

     https://thesource2.metro.net/i/document/K0W1U2/radar-principles-for-the-nonspecialist_pdf 

     https://digital-library.theiet.org/content/books/ra/sbra032e 

     Personally, I try to keep abreast of nice new discussions or revelations in radar:

     https://us.artechhouse.com/Radar-for-Fully-Autonomous-Driving-P2262.aspx  (This new book is highly recommended for what it contains!)

     https://hal.archives-ouvertes.fr/hal-01070959/document 

     https://www.facebook.com/IEEEAESS/videos/passive-radar/748838592565039/ 

    https://www.facebook.com/IEEEAESS/videos/radar-adaptivity-antenna-based-signal-processing-techniques/337163170947463/ 

For aircraft and other air-borne or space-borne radar targets, it is important to know the effect of fluctuations, as contained in information regarding 
whether it is a Marcum type or Swerling type I to V target. https://en.wikipedia.org/wiki/Fluctuation_loss (Both Marcum and Swerling were at RAND when they developed these classifications in the 1950's.)
See more details regarding Swerling targets in https://apps.dtic.mil/sti/pdfs/AD0080638.pdf 

Useful tools for handling actual data for the application: Julian-to-local-time conversions and Calendar conversions.

Elaborating further on TK-MIP Version 2.0 CURRENT CAPABILITIES: This Software package allows/enables the USER to:

·      SIMULATE any Gauss-Markov process specified in linear time-invariant (or time-varying) state space form of arbitrary dimension N (usually N is less than 200 and frequently much less).
[In order to perform simulations as either just a single sample function or Monte-Carlo, the transition matrix utilized is usually calculated using some form of the Matrix Exponential using standard practical numerical computations. Historically, we at TeK Associates encountered an error in the documented calculation of the Matrix Exponential related to its stopping condition that is used in determining the number of terms retained in the approximating Taylor series to attain a prescribed accuracy [74]. We modestly corrected another prevalent error [73] without making "waves" so to speak by not identifying the specific coder or their organization.] The time-varying case can be handled incrementally by decomposing the time epoch of interest to consist of smaller time intervals, within which the system may again be validly approximately as time-invariant. Alternatively, USER can chose the option solving underlying Ordinary Differential Equation (ODE) using predictor-corrector integration. Click the following link to see the detailed options offered within TK-MIP: http://www.tekassociates.biz/reconfigurable_s_w.htm (this is a pointer to structural options that can be tailored within TK-MIP to match the application's structure by merely clicking the appropriate option within TK-MIP). The time-varying case can be handled incrementally by decomposing the time epoch of interest to consist of smaller time sub-intervals, where the system is again validly approximated by one that is time-invariant. Alternatively, USER can chose the option for solving the underlying ODE using predictor-corrector integration or Euler integration.

·      TEST for WHITENESS and NORMALITY of SIMULATED Gaussian White Noise (GWN) sequence: w(k) for k=1, 2, 3,... (ultimately from underlying Pseudo-Random Number [PRN] generator).

·      SIMULATE sensor observation data as the sum of a Gauss-Markov process, C x(k), and an INDEPENDENT Gaussian White Measurement Noise, G v(k).

·      DESIGN and RUN a Kalman FILTER with specified  gains, constant or propagated, on REAL (Actual) or SIMULATED observed sensor measurement data.

·        DESIGN and RUN a Kalman SMOOTHER (now also known as Retro-diction) or a Maximum Likelihood Batch Least Squares algorithm on REAL (actual) or SIMULATED data.

·        DISPLAY both DATA and ESTIMATES graphically in COLOR  and\or output to a printer or to a file (in ASCII).

·        CALCULATE SIMULATOR and ESTIMATOR response to ANY (Optional) Control Input.

·        EXHIBIT the behavior of the Error Propagation (Riccati) Equation:

·        For full generality, unlike most other Kalman filter software packages or add-ons like MATLAB® with SIMULINK® and their associated Control Systems toolkit, TK-MIP avoids using Schur computational techniques, which are only applicable to a narrowly restricted set of time-invariant system matrices and TK-MIP® also avoids invoking the Matrix Signum function for any calculations because they routinely fail for marginally stable systems (a condition which is not usually warned of prior to invoking such calculations within MATLAB®). See Notes in Reference [2] at the bottom of this web page for more perspective.

·        From a current session either the final result or incremental intermediate result having just been completed, USER can SAVE-ALL at END of each Major Step (to gracefully accommodate any external interruptions imposed upon the USER) or RESURRECT-ALL results previously saved earlier, even from prior sessions. [This feature is only available for the simpler situation of having  linear time-invariant (LTI) models and not for time-varying models, nor for nonlinear situations, nor for Interactive Multiple Model (IMM) filtering situations, all of which can be handled within TK-MIP after slight modifications (as directed by the USER from the MAIN MENU, Model Specification, and Filter Processing screens) but these more challenging scenarios do NOT offer the SAVE-ALL and RESURRECT-ALL capability because of the additional layers of complexity to be encountered for these special cases causing them to be less amenable to being handled within a single structural form].

·      OFFER EXACT discrete-time EQUIVALENT to continuous-time white noise (as a USER option) for greater accuracy in SIMULATION (and closer agreement between underlying model representation in FILTERING and SMOOTHING).

·       AVAIL the USER with special TEST PROBLEMS\TEST CASES (of known closed-form solution) to confirm PROPER PERFORMANCE of any software of this type.

·       OFFER ACCESS to time histories of User-designated Kalman Filter\Smoother COVARIANCES (in order to avail a separate autonomous covariance analysis capability). Benefit of having a standard Covariance Analysis capability is that it can be used to establish Estimation Error Budget Specifications (before systems are built). Another theoretical approach to obtaining this is here and here.

·       ACCEPT output transformations to change coordinate reference for the Simulator, the Filter state OUTPUTS and associated Covariance OUTPUTS [a need that arises in NAVIGATION and Target Tracking applications as a User-specified time-varying orthogonal transformation with User-specified coordinate off-set (also known as possessing a specified time-varying bias)]. TK-MIP supplies a wide repertoire of pre-tested standard coordinate transforms for the USER to choose from or to concatenate to meet their application needs (which avoids the need to insert less well validated USER code here for this purpose). 

·       OFFER Pade Approximation Technique as a more accurate alternative (for the same number of terms retained) to use of standard Taylor Series approach for calculating the Transition Matrix or Matrix Exponential.

·        PERFORM Cholesky DECOMPOSITION (to specify F and\or G from specified Q and\or R) as

Q = F · F^T,

      and as 

R = G · G^T,

      where outputted decomposition factors F^T and G^T are upper triangular.

·        Cholesky can also be used in an attempt to investigate a matrix’s positive definiteness\ semi-definiteness (as arises for Q, R, P₀, P, and M [defined further below]).

·        PERFORMS Matrix Spectral Factorization (to handle any serially time-correlated noises encountered in application system modeling by putting them in the standard KF form via state augmentation) [e.g., In the frequency domain, the known associated matrix power spectrum is factored to be of the form

S_ww(s) = W^T(-s) · W(s),

      (where s is the bilateral Laplace Transform variable) then one can perform a REALIZATION from one of the above output factors as

 W^T(-s) = C₂ (sI_nxn-A₂)^-1 F₂,

      to now accomplish a complete specification of the three associated subsystem matrices on the right hand side above (where both Matrix Spectral Factorization and specifying system realizations of a specified Matrix Transfer Function of the above form are completely automated within TK-MIP). The above three matrices are used to augment the original state variable representation of the system as [C|C₂], [A|A₂], [F|F₂] so that the newly augmented system (now of a slightly higher system state dimension) ultimately has only WGN contaminating it as system and measurement noises (as again putting the associated resulting system into the standard form to which Kalman filtering directly applies).]

·        OUTPUT results:

*         to the MONITOR Screen DISPLAY,

*         to the PRINTER (as a USER option) and can have COMPRESSED OUTPUT by sampling results at LARGER time steps (or at the SAME time step) or for fewer intermediate variables,

*         to a FILE (ASCII) on the hard disk (as a USER option) [separately from SIMULATOR, Kalman FILTER, Kalman SMOOTHER (for both estimates and covariance time histories)].

·       OUTPUTS final Pseudo Random Number (PRN) generator seed value so that if subsequent runs are to resume, with START TIME being the same as prior FINAL TIME, the NEW run can dovetail with the OLD as a continuation of the SAME sample function (by using outputted OLD final seed for PRN as NEW starting seed for resumed PRN).

·        SOLVE FOR Linear Quadratic Gaussian\Loop Transfer Recovery (LQG\LTR) OPTIMAL Linear feedback Regulator Control for ONLY the LTI case [of the following feedback form, respectively, involving either the explicit state or the estimated state, depending upon which is more conveniently available in the particular application], either as:

      or as

      for both by utilizing a planning interval forward in time either over a FINITE horizon or over an INFINITE horizon cost index (i.e., as transient or steady-state cases, respectively, where M in the above is constant only for the steady-state case). Strictly speaking, only the last expression is (the less benign, more controversial) LQG or (more benign) LQG\LTR since the former expression is (more benignly) LQ feedback control (that is always stable unlike the situation for pure, unmitigated LQG, which lacks sufficient gain margin or sufficient phase margin).

·      Provide Password SECURITY capability, compliant with the National Security Administration’s (NSA’s) Greenbook specifications (Reference [3] at the bottom of this web page) to prevent general unrestricted access to data and units utilized in applications in TK-MIP that may be sensitive or proprietary. It is mandatory that easy access to outsiders be prevented, especially for Navigation applications since enlightened extrapolations from known gyro drift-rates in airborne applications can reveal targeting, radio-silent rendezvous, and bombing accuracy’s [which are typically Classified]; hence critical NAV system parameters (of internal gyros and accelerometers) are usually Classified as well, except for those that are so very coarse that they are of no interest in tactical or strategic missions.

·     Offer information on how the USER may proceed to get an appropriate mathematical model representation for common applications of interest by offering concrete examples & pointers to prior published precedents in the open literature, and provide pointers to third party software & planned updates to TK-MIP to eventually include this capability of model inference/model-order and structure determination from on-line measured data. Indeed, a journal to provide such information has begun, entitled Mathematical Modeling of Systems: Methods, Tools and Applications in Engineering and Related Sciences, Swets & Zeitlinger Publishers, P.O. Box 825, 2160 SZ LISSE, Netherlands (Mar. 1995).

Click here to get the (draft copy) of AIAA Standards for Atmospheric Modeling, as specified by the various U.S. and other International agencies.  

·       Click this link to view the TK-MIP ADVANCED TOPICS (and some of its options). To return to this point afterwards to resume reading, merely use the back arrow at the top left of your Web Browser or the keyboard: ALT + Left Arrow.

Dr. Paul J. Cefola, the expert referenced above, has a consultancy in Sudbury, Massachusetts: cefola “at” comcast.net.

·      Depicts other standard alternative symbol notations (and identifies their source as a precedent) that have historically arisen within the Kalman filter context and that have been adhered to (for awhile, anyway) by pockets of practitioners. This can be an area of considerable confusion, especially when notation has been standardized for decades, then modern writers (possibly unaware of the prior standardization or, more likely, opting to ignore it) use different notation in their more recent textbooks on the same subject, thus driving a schism between the old and new generation of practitioners. “A rose by any other name smells just as sweet.”- From Shakespeare's Romeo and Juliet

·       Provides a mechanism for our “shrink-wrap” TK-MIP software product to perform Extended Kalman Filtering (EKF) and be compatible with other PC-based software (accomplished through successful cross-program or inter-program communications and hand-shaking and facilitated by TeK Associates recognizing and complying with existing Microsoft Software Standards for software program interaction such as abiding by that of ActiveX or COM). Therefore, TK-MIP® can be used either in a stand alone fashion or in conjunction with other software for performing the estimation and tracking function, as indicated in our symbolic animation below: