 Research
 Open Access
Timehomogeneous Markov process for HIV/AIDS progression under a combination treatment therapy: cohort study, South Africa
 Claris Shoko^{1}Email author and
 Delson Chikobvu^{1}
https://doi.org/10.1186/s1297601700754
© The Author(s). 2018
 Received: 25 July 2017
 Accepted: 6 December 2017
 Published: 18 January 2018
Abstract
Background
As HIV enters the human body, its main target is the CD4 cell which it turns into a factory that produces millions of other HIV particles. These HIV particles target new CD4 cells resulting in the progression of HIV infection to AIDS. A continuous depletion of CD4 cells results in opportunistic infections, for example tuberculosis (TB). The purpose of this study is to model and describe the progression of HIV/AIDS disease in an individual on antiretroviral therapy (ART) follow up using a continuous time homogeneous Markov process. A cohort of 319 HIV infected patients on ART follow up at a Wellness Clinic in Bela Bela, South Africa is used in this study. Though Markov models based on CD4 cell counts is a common approach in HIV/AIDS modelling, this paper is unique clinically in that tuberculosis (TB) coinfection is included as a covariate.
Methods
The method partitions the HIV infection period into five CD4cell count intervals followed by the end points; death, and withdrawal from study. The effectiveness of treatment is analysed by comparing the forward transitions with the backward transitions. The effects of reaction to treatment, TB coinfection, gender and age on the transition rates are also examined. The developed models give very good fit to the data.
Results
The results show that the strongest predictor of transition from a state of CD4 cell count greater than 750 to a state of CD4 between 500 and 750 is a negative reaction to drug therapy. Development of TB during the course of treatment is the greatest predictor of transitions to states of lower CD4 cell count. Transitions from good states to bad states are higher on male patients than their female counterparts. Patients in the cohort spend a greater proportion of their total followup time in higher CD4 states.
Conclusion
From some of these findings we conclude that there is need to monitor adverse reaction to drugs more frequently, screen HIV/AIDS patients for any signs and symptoms of TB and check for factors that may explain gender differences further.
Keywords
 HIV/AIDS progression
 Homogeneous Markov models
 Reaction to treatment
Background
The life cycle of HIV starts as it enters the human body, its major target being a white blood cell called Thelper cells or CD4 cells [1]. Once these cells are infected, HIV takes over and turns them into factories that produce thousands of copies of the virus. The HIV makes use of the enzyme Reverse Transcriptase to change copies of its Ribonucleic Acid (RNA) into Deoxyribose nucleic Acid (DNA). The viral DNA then enters the nucleus of the host cell and combines with cell DNA and starts making copies of viral RNA. The enzyme protease helps in assembling the viral particles into thousands of new viruses, which will bud and destroy the host cell. These new viruses will then be ready to attack other CD4 + T cells. Hence, the importance of CD4+ T cell count in understanding the progression of HIV. Depreciation of the CD4+ T cells in the human body leads to the deterioration of the human immune system, which is why the virus is called the Human Immunodeficiency Virus (HIV).
As the immune system is compromised, the individual is now prone to opportunistic infections like tuberculosis (TB). TB may occur at any stage of HIV disease and is frequently the first recognised presentation of underlying HIV [2]. HIV and TB coinfection is characterised by challenges that include poor adherence and overlapping toxicities resulting in an impaired CD4 Tcell recovery with antiretroviral therapy (ART) due to the effect of drugdrug interaction [3].
The need to address the challenges associated with HIV/AIDS progression in the presence of TB coinfection has prompted this study and also to analyse HIV disease history based CD4 multistates and death/loss to followup in a single model. However, most studies use KaplanMeier analysis and Cox proportional hazards regression models [3–5]. KaplanMeier survival methods and Cox proportional hazards regression are commonly employed tools to model mortality and time to viral suppression and/or subsequent rebound and occasionally used to model time to CD4 recovery. However, survival models are not appropriate for all studies, particularly in the presence of competing risks and when multiple or recurrent outcomes are of interest. In particular, when modelling HIV/AIDS progression, Markov models are relatively straight forward to analyse both CD4 stage and death or loss to followup within a single model which survival models fail to do. Markov models can accommodate censored data, competing risks (informative censoring), multiple outcomes, recurrent outcomes, frailty and nonconstant survival probabilities [6]. Examination of the conditions of the stochastic processes at various points in time, categorisation of the conditions, and examination of the external influences on the stochastic processes can be done using Markov models [7]. Markov models are favourable to the modelling of diseases in particular cases where the disease is grouped into a set of exhaustive and mutually exclusive health states, thereby forming a multistate model [8]. History is naturally generated as the multistates evolve over time; it contains information on previous visits, time of entry into various states, and the length of stay in states.
Continuoustime homogeneous Markov models have been used since early in the epidemic to model disease progression of HIV/AIDS patients, and there has been some recent renewed interest in the use of these models. In 1989 Longini et al. used a 5state Markov model based on the clinical indicators of the HIV disease progression [9]. In 1998 Alioum et al. estimated the effects of gender, age, mode of transmission and ART on HIV progression using a 3state Markov model [10]. In 2011 Reddy [11] carried out a research almost similar to Alioum et al. in South Africa. However, Reddy used a 5state Markov model with 4 CD4 based transient states followed by the absorbing state, ARV initiation. Reddy’s model is characterised by high rates of immune deterioration since the study was carried out on ARV naïve patients. In 2009, Binquet et al. used a multistate Markov model to analyse the impact of gender, intravenous drug use, weight loss, low haemoglobin, CD8 cell count and HIV viral load on HIV evolution in the era of highly active antiretroviral therapy (HAART) [12]. Recently, in 2013 Grover et al. assessed the impact of ART using a 5staged multistate Markov model and went further to examine the effects of explanatory variables; age, sex and mode of transmission on the transition rates [13].
In this study, we use 7staged continuoustime Markov model to assess the disease progression of HIV/AIDS patients receiving ART from a clinic in BelaBela, South Africa. The first 5 stages are based on CD4 cell counts and the end points are either death or withdrawal from study. In addition to the gender and age differences in HIV/AIDS progression, we further assess the effects of having TB as the initial marker of HIV/AIDS, developing TB during the course of treatment, developing some adverse effects to treatment (Reaction), CD4 baseline and viral load baseline. Though Markov models based on CD4 cell counts is a common approach in HIV/AIDS modelling, this paper is unique clinically in that tuberculosis (TB) coinfection is included as a covariate.
In medical research, the state of the patient at observation time is the only thing known with certainty. The researcher may know the time interval in which a transition has occurred, but not the exact time. Thus, homogeneous Markov models which are interval censored can handle such data [14]. The transition intensities, probabilities and the distribution functions associated with the times are the basic building blocks of the Markov processes [15]. For a continuoustime Markov model, transitions can occur at any (realvalued) time instant. For a timehomogeneous Markov jump process, the holding time in state i are modelled using exponential distributions. The exponential distributions may be adequate for many reallife situations, for example, time until death, and waiting time before moving to another state. However, the exponential distributions are memoryless continuous distributions, hence a limitation in the application of Markov processes. The ‘memoryless’ property could be seen as a problematic assumption in this setting. It is likely the case that patients starting on ART who respond well to treatment will continue to respond well to treatment  contradicting the Markov assumption and memoryless property.
In this paper we describe, using the theory of continuous time Markov processes, and using real data on an evolving disease such as AIDS. Also, the effects of covariates, including TB, on baseline transition rates is considered. Models with and without covariates are fitted and compared using the likelihood ratio test.
The next section explores the methods of Markov modelling and an illustrative case study on HIV progression is given. In this section, data used in the analysis is described and we explain formulation of the model based on the data. This is followed by a section on the results and discussions. The final section concludes on the findings.
Methods
A continuoustime homogeneous Markov model
α_{ ij }, for i = 1, …, 5 and j = 1, …, 7, does not vary over time and satisfies the following conditions; \( {\sum}_{j\in X}{\alpha}_{ij}=0 \) and \( {\alpha}_{ii}={\sum}_{i\ne j}{\alpha}_{ij} \).
where k is a state that the system can pass through as it makes a transition from state i to state j. The time homogeneous models are fitted for this data to assess effectiveness of the treatment by comparing the forward transition and the reverse transitions. This then lead to building of models that allow transitions in both directions.
Data description
The model is initially applied on 319 HIV patients on antiretroviral therapy (ART) from a Wellness clinic in Bela Bela, South Africa, from year 2005 to year 2009. Two hundred and twentyseven of these patients were females and 92 were males at treatment commencement (t = 0). After 3 years of treatment uptake, 173 females and 71 males were remaining in the study. Thirtyeight females had died and 16 withdrew and their status was not known after 3 years of treatment up take. Nineteen of the males died during the first three years and two had withdrawn and it was not known whether they were alive or dead. A 2year old (subject 81) together with subject 82 were detected by the residuals plot as an outlier and it was removed from the analysis meaning that the remaining 317 patients were used for analysis. About 50 and 65% of the female and male deaths respectively occurred during the first 6 months of treatment uptake. The interquartile range of patient ages is (33; 47.5) years with a mean and median age of 39.53 and 40 years respectively. The ages were negatively skewed (skew = −0.24) which means that there were more younger patients than older patients in this cohort.
At time t = 0 there were 242 individuals with CD4 baseline (CD4BL) cell count below 200, 59 individuals with CD4 cell count between 200 and 350, 11 individuals with CD4 cell count between 350 and 500, 6 individuals with CD4 cell count between 500 and 750 and 1 individual with CD4 cell count above 750. At t = 0 the CD4 cell count had mean of 156 cells/mm^{3}, a median of 116 cell/mm^{3} and the maximum CD4 cell count was 1202 cells/mm^{3}. The mean viral load baseline (VLBL) for these patients was 105,573.35 copies/mm^{3} and it ranged from 56 to 818,600 copies/mm^{3}. The median viral load was 58,523.00 copies/mm^{3}. From these individuals 155 had a WHO stage baseline (WSBL) of 4 which is related to severe HIV symptoms. WSBL is the categorisation of HIV/AIDS at baseline basing on the clinical markers as defined by World Health Organisation (WHO).
Although some individuals developed TB (DTB) during the course of treatment, 109 patients had TB as an initial marker of HIV. From the individuals who had TB before (TBB4) commencement of antiretroviral therapy (ART), 66 had a CD4 baseline below 200cells/mm^{3}, 20 had a CD4 baseline between 200 and 350cells/mm^{3}, 2 had CD4 baseline between 350 and 500cells/mm^{3}, 2 between 500 and 750cells/mm^{3} and 19 had unknown CD4 baseline. These patients completed their TB treatment before commencement of ART. Fiftytwo patients developed TB during the treatment period and 12 of these patients had TB before commencement of treatment. During the first 6 months of treatment uptake, 35 patients died and from these deaths, only five were attributed to having TB before commencement of ART.
A combination therapy was administered to all HIVinfected individual in the cohort. The therapeutic intervention inhibits the actions of reverse transcriptase enzyme and/or protease of new infectious free HIV by the HIVinfected cell. The drug regimens at t = 0 were mainly a combination of d4T3TCEFV (administered to 207 patients) and d4T3TCNVP (administered to 83 patients). The second line regimens were mainly a combination of AZT3TCEFV/NVP and were given to patients who developed some adverse reaction. These second line regimens were frequently used from 2 to 4 years posttreatment commencement. The therapeutic intervention lowers the number of infectious free virus particles in the circulation, and in some cases to beyond detection. This results in a reduction on the density of infected cells, causing a rise on the CD4 cell count of infected individuals. So generally the CD4 cell count of an individual receiving therapeutic intervention is expected to rise to well above 500 cell/mm^{3}, assuming a proper adherence to treatment. Hence the use of increase in CD4 cell count as the marker of efficacy of treatment.
During the course of treatment, some individuals developed some adverse reaction (React) to treatment. For these individuals the adverse reactions were treated and drugs administered to them were changed. Change of treatment was also based on the viral load monitoring.
WSBL  Gender  Age  CD4BL  VLBL  DTB  TBB4  React  

1  4  Male  ≤ 40years  ≤ 350  ≥10 000  Yes  Yes  Yes 
0  other  Female  > 40years  > 350  < 10 000  No  No  No 
Model formulation
State 1CD4 ≥ 750  State 2–500 ≤ CD4 < 750 

State 3–350 ≤ CD4 < 500  State 4–200 ≤ CD4 < 350 
State 5CD4 < 200  State 6Death 
State 7Withdrawal. 
The information in Fig. 1 shows that state 6 and 7 are absorbing states hence no transitions from these states. As HIV progresses in an individual’s body, there is a possibility of an individual being in the same state in consecutive visit times.
Transition Counts from 2005 to 2009
To  1  2  3  4  5  6  7  
From  1  69  34  5  1  1  1  1 
2  47  80  37  4  0  2  4  
3  21  79  193  46  9  6  5  
4  0  14  128  203  37  8  9  
5  0  3  26  117  204  42  8 
Table 1 shows that, transition counts from state i to i ± j are higher for all the values in which j = 1 than for j > 1 where i, j ∈ { 1, …, 5}. As a result a bidirectional model is proposed which defines transitions from state i to i ± 1 or from i to j = 6; 7.

CD4 cell count of an individual is expected to rise due to efficacy of treatment at a rate of α_{ ij }, where j = i − 1;

Some individuals fail to adhere to treatment therapy. These individuals can move to a state of lower CD4 cell count at a rate of α_{ ij }, where j = i + 1;

From any state i = 1, 2, …, 5 an infected individual can die (state 6) at a rate of α_{i6};

An individual in state i = 1, 2, …, 5 can decide to withdraw (state 7) at a rate of α_{i7};

An individual can remain in the same state at a rate of α_{ ii } = − λ_{ i } = − (α_{i, i − 1} + α_{i, i + 1} + α_{i6} + α_{i7}). This is based on the fact that the sum of transition rates from any state is equal to zero.
Equations (4) to (10) represent all the possible transition probabilities from state i, for i = 1, 2, ...5, to state j = 1, …, 7. p_{ ij }(t) represents the probability that a patient in state i makes a transition to state j and its coefficients represent the transition rates. For example, in equation (4), −(α_{12} + α_{16} + α_{17}) = α_{11}. These states denoted by i are defined based on the CD4 cell count grouping. So there is a possibility of a backward or forward movement transition between transient states due to failure or efficacy of treatment respectively. There is no possible transition from state i = 6 and state i = 7 because these states are absorbing states where i = 6 represents death of an infected individual and state i = 7 represents withdrawal from treatment by an infected individual. All the analysis is done using the package ‘msm’ for multistate modelling in R software. The package was developed by Jackson in 2011 [16].
Results and discussions
Estimation of the transition rate matrix
where t_{i } : i = 1, 2, …, 7, is the total number of observed waiting/holding time in state i, \( {\alpha}_{ii}={\sum}_{i\ne j}{\alpha}_{ij} \) and n_{ ij } is the number of transitions observed from state i to state j. The estimates are obtained by taking the logarithm of the likelihood and differentiating this with respect to each of the transition intensities α_{ ij }'s. This leads to the maximum likelihood estimates of the transition intensities as \( {\alpha}_{ij}=\frac{n_{ij}}{t_i} \), where n_{ ij } is the number of transitions from state i to state j, and t_{ i } the total observed waiting/holding time in state i.
The plot of residuals for each of the individuals in the study was drawn to identify the outliers (subjects with higher influence) in the data. Once the outliers are identified they can simply be deleted and the model is refit. According to Titman in 2007 [17] residuals for multistate models can be determined as follows;
Transition intensities and their corresponding confidence intervals for the model with and the model without outliers
With outliers  Without outliers  

α _{12}  0.9820 (0.6695,1.4410)  0.8872 (0.6355,1.2390) 
α _{16}  0.1176 (0.0678,0.2040)  0.1016 (0.0541,0.1907) 
α _{17}  0.1153 (0.0663,0.2005)  0.1418 (0.0843,0.2386) 
α _{21}  0.6871 (0.4814,0.9808)  0.5183 (0.3723,0.7217) 
α _{23}  0.4256 (0.3129,0.5790)  0.4959 (0.3729,0.6593) 
α _{26}  0.0726 (0.0406,0.1300)  0.0811 (0.0468,0.1404) 
α _{27}  0.0261 (0.0093,0.0734)  0.0377 (0.0167,0.0850) 
α _{32}  0.4373 (0.3605,0.5305)  0.4231 (0.3451,0.5186) 
α _{34}  0.3382 (0.2639,0.4335)  0.3324 (0.2605,0.4241) 
α _{36}  0.0273 (0.0127,0.0587)  0.0145 (0.0047,0.0444) 
α _{37}  0.0298 (0.0155,0.0574)  0.0311 (0.0166,0.0584) 
α _{43}  0.5524 (0.4693,0.6503)  0.5183 (0.4389,0.6120) 
α _{45}  0.2242 (0.1686,0.2980)  0.2523 (0.1936,0.3287) 
α _{46}  0.0361 (0.0171,0.0763)  0.0033 (0.0149,0.0742) 
α _{47}  0.0382 (0.0225,0.0647)  0.0541 (0.0352,0.0833) 
α _{54}  0.5482 (0.4651,0.6463)  0.5164 (0.4356,0.6123) 
α _{56}  0.0904 (0.0622,0.1316)  0.0906 (0.0625,0.1313) 
α _{57}  0.0357 (0.0211,0.0605)  0.0288 (0.0158,0.0524) 
2xLL  3969.72  3941.971 
The results are shown in Appendix 2 and 3 for CD4BL and WHOSBL respectively. The results from Appendix 2 show that transition rates to CD4 recovery (2 to 1, 3 to 2, 4 to 3 and 5 to 4) were high for patients who initiated therapy when their CD4 baseline level was well above 350 per mm^{3}. These rates of CD4 recovery decrease with as the CD4 cell count at treatment initiation decrease with a baseline CD4 cell count below 200 per mm^{3} recording the lowest rates of CD4 recovery. The results from Appendix 3 show that regardless of the WHO stage baseline, transition rates to CD4 recovery are higher than transition rates to CD4 deterioration. The rates of CD4 recovery are the highest for transitions from state 5 to state 4. Transition rates to state 6 (death) are the highest for those individuals who had severe HIV symptoms (WHOBL = 4) and these intensities decrease as the symptoms decrease from severe to asymptomatic levels.
Expected holding times
Expected holding times in each state
i  Estimates  SE  L  U 

1  0.8844741  0.12602569  0.6689603  1.169418 
2  0.8826571  0.09158707  0.7202261  1.081721 
3  1.2482706  0.09636263  1.0729973  1.452175 
4  1.2077295  0.07925375  1.0201303  1.331720 
5  1.5728163  0.11539878  1.3621492  1.816065 
Results from Table 3 show estimates of the holding time, the standard error (SE), the lower bound (L) and the upper bound (U) for each of the transient state i. From the results, if an individual is in state 5 (corresponding to a CD4 count below 200cell/mm^{3}) he spends more time in that state before making a transition to other states. This could be due the time taken by an individual to respond to treatment since state 5 is the worst state in HIV/AIDS progression.
The jump chain
Probability of each State being next (R_{ ij })
To  1  2  3  4  5  6  7  
From  1  0  0.7847  0  0  0  0.0899  0.1254 
2  0.4575  0  0.4377  0  0  0.0715  0.0333  
3  0  0.5282  0  0.4149  0  0.0181  0.0388  
4  0  0  0.6041  0  0.294  0.0388  0.0631  
5  0  0  0  0.8123  0  0.1425  0.0451  
6  0  0  0  0  0  1  0  
7  0  0  0  0  0  0  1 
The results from Table 4 show that R_{i, 1 − 1} > R_{i, i + 1}, which shows that the probability of jumping to a better state is higher than the probability of jumping to a worse state. This is more pronounced for individuals in state 5 where the probability of jumping to state 4 (recovery) is 0.8123 which is very high compared to probability of making a jump to state 6. This is an indication of the effectiveness of treatment. Probability of the death state being next is the highest for those patients with CD4 counts less than 500. These probabilities increase with the decreasing number of CD4 counts.
Forecast of the total length of stay in each state
State1  State2  State3  State4  State5  State6  State7 

8.988960  8.806075  7.767124  3.520485  1.153648  Inf  Inf 
The results show that each individual is forecasted to spend approximately 8.99 half years in state 1, 8.8 half years in state 2, 7.77 half years in state 3, 3.52 half years in state 4 and finally 1.153 half years in state 5. These results show that HIV positive individuals on treatment are expected to spend more time in good states compared to the time spent in bad states.
Percentage prevalence for the model without covariates.
The results from Fig. 3 show that for the state i = 1, …, 6 the expected prevalence fit the observed data perfectly well except for the withdrawal state where the expected prevalence overestimate the observed. The plots further show a sharp decrease on state 5 percentage prevalence with the fitted model, underestimating the model for observed data up to time = 7 half years. The percentage prevalence for the death state is increasing at a slow rate and from time = 2 half years to time = 8 half years the percentage prevalence is stable.
Effects of covariates on transition intensities
Baseline intensities and their corresponding confidence intervals for the covariate effects
(i, j)  Intensities (α_{ ij })  B.L. Intensities \( \left({\alpha}_{ij}^{(0)}\right) \) 

(1, 2)  0.8872 (0.6355,1.2390)  0.5030 (0.4277,0.8612) 
(1, 6)  0.1016 (0.0541,0.1907)  0.0200 (0.0100,0.4641) 
(1, 7)  0.1418 (0.0843,0.2386)  0.0175 (0.009,0.4934) 
(2, 1)  0.5183 (0.3723,0.7217)  0.3900 (0.3846,0.6863) 
(2, 3)  0.4959 (0.3729,0.6593)  0.4440 (0.2973,0.5550) 
(2, 6)  0.0811 (0.0468,0.1404)  0.0111 (0.0073,0.2018) 
(2, 7)  0.0377 (0.0167,0.0850)  0.0116 (0.001,0.0189) 
(3, 2)  0.4231 (0.3451,0.5186)  0.3760 (0.3252,0.5010) 
(3, 4)  0.3324 (0.2605,0.4241)  0.2333 (0.2076,0.3641) 
(3, 6)  0.0145 (0.0047,0.0444)  0.0063 (0.00367,0.1287) 
(3, 7)  0.0311 (0.0166,0.0584)  0.0095 (0.00285,0.3149) 
(4, 3)  0.5183 (0.4389,0.6120)  0.5600 (0.4485,0.6284) 
(4, 5)  0.2523 (0.1936,0.3287)  0.2300 (0.1522,0.2785) 
(4, 6)  0.0033 (0.0149,0.0742)  0.0070 (0.0049,0.4210) 
(4, 7)  0.0541 (0.0352,0.0833)  0.0084 (0.00589,0.1547) 
(5, 4)  0.5164 (0.4356,0.6123)  0.5020 (0.4450,0.6297) 
(5, 6)  0.0906 (0.0625,0.1313)  0.0198(0.00662,0.1456) 
(5, 7)  0.0288 (0.0158,0.0524)  0.0055 (0.001280,3.936) 
2xLL  3941.971  3699.259 
−2logLR  Df  pvalue  

with.covariates  1770.186  144  10^{−4} 
The results show that the model with covariates fits significantly better than the model without covariates.
Hazard ratios of covariates on transition intensities
Hazard ratios for the covariates on intensities
(i, j)  VLBL  DTB  TBB4  Gender  React  CD4BL  WSBL  Age 

(1, 2)  0.69  2.29  0.31  2.04  4.72  1.17  0.55  1.45 
(1, 6)  1.57  0.87  1.11  1.55  0.60  1.30  1.11  0.83 
(1, 7)  1.01  0.96  1.00  1.26  0.68  0.92  0.56  0.40 
(2, 1)  0.48  1.14  0.76  1.33  1.46  1.17  0.74  2.63 
(2, 3)  0.92  1.98  0.67  6.46  0.67  1.37  0.26  0.40 
(2, 6)  1.12  0.96  1.69  1.45  0.54  0.997  1.16  1.10 
(2, 7)  1.30  1.19  1.92  0.93  0.73  1.33  0.79  0.70 
(3, 2)  0.68  1.58  1.10  2.35  2.08  0.69  0.75  0.83 
(3, 4)  0.42  2.55  0.53  1.04  0.55  1.57  1.06  0.89 
(3, 6)  0.92  1.17  2.03  1.20  0.59  1.05  1.32  1.77 
(3, 7)  1.34  0.86  2.34  0.72  0.20  1.20  0.54  2.06 
(4, 3)  1.52  1.72  0.86  0.61  1.02  0.27  0.84  1.08 
(4, 5)  0.74  2.25  1.65  1.36  0.65  1.05  0.46  0.89 
(4, 6)  1.31  1.09  1.09  1.59  0.22  1.01  1.40  1.96 
(4, 7)  1.56  1.17  1.88  0.58  0.36  1.02  0.52  2.18 
(5, 4)  0.51  1.86  1.02  0.81  1.32  0.61  0.40  0.63 
(5, 6)  0.92  0.65  0.41  2.10  0.06  1.003  2.09  2.60 
(5, 7)  0.97  1.20  0.88  1.19  0.66  0.87  0.40  2.31 
The 2xLL for the model fitted in Table 6 is 3699.259. The results show that the strongest predictor of transition from state 1 to 2 is a negative reaction to treatment, which has a hazard ratio of 4.715. This means that patients who developed some form of reaction were over 4 times more likely to transit from a level of CD4 ≥ 750 to a level of 500 ≤ CD4 < 750 than patients who did not react to treatment. However, from all the other states, hazard ratios for the patients who reacted to treatment are higher for immune recovery than for immune deterioration.
The strongest predictor of immune deterioration from a CD4 level between 350 and 500 to a CD4 level between 200 and 350 (3 to 4) is developing TB during treatment, with a hazard ratio of over 2. Developing TB is also the strongest predictor of immune deterioration from 4 to 5, with a hazard ratio also greater than 2. This means that TB is the major cause of further immune deterioration when the immune system is too weak. Hence the recommendation that HIV patients should continuously have their TB status checked. Those individuals who had TB before enrolment had the strongest predictor for the transition from state 3 to state 6. These patients had a hazard ratio of over 2 times more likely to die from state 3 than those who were enrolled without having TB. However, for these individuals, transitions to better states were generally higher than transitions to worse states for almost all states.
A hazard ratio of 6.46 for the predictor variable male shows that males were over 6 times more likely to transit from state 2 to 3 than their female counterparts. The hazard ratios of males from a bad state to a better state are less than 1, which is an indication that males are less likely to respond to treatment compared to females.
The hazard ratios for the transitions to a better state for patients who were enrolled with CD4 counts below 350 are less than one, but hazards to worse states are greater than one, an indication that starting treatment when the CD4 levels are below 350 retards immune recovery. The transitions to the death state for individuals who started treatment when they were on the WHO stage of 4 are all more than one, meaning that starting treatment with a WHO stage of 4 is a leading cause of being absorbed in the death state.
Percentage prevalence for the model with covariates
Figure 4 confirms that the inclusion of covariates on the model improves the fitness of the model since the expected prevalence is now perfectly closer to the observed prevalence for all states than the model without covariates.
Conclusion
In this paper a continuoustime homogeneous Markov model is fitted to explore predictors of HIV/AIDS progression for patients on antiretroviral therapy. A continuoustime homogeneous model is fitted with and without covariates and comparison of these two models is done using the likelihood ratio test. Parameters that define progression of HIV/AIDS were estimated and these include transition intensities, mean sojourn times and probability of each state being next or jump chains. The fitted model is used to analyse the effects of the covariates on the transition intensities. These covariates were reaction to treatment, development of TB during treatment and gender among others.
Results from the likelihood ratio test show that the model with covariates provides a better fit than the model with no covariates with a pvalue =10^{{−4}}. The results show that transition rates to immune recovery are generally higher than the transition rates to immune deterioration. However, the results show that the strongest predictor of immune deterioration from state 1 (CD4 cell count greater than 750) to state 2 (CD4 cell count between 500 and 750) is reaction to treatment. These patients are 4 times more likely to transit from state 1 to state 2 than those who did not react to treatment.
Patients who developed TB during the course of treatment have higher chances of immune deterioration than immune recovery compared to those who did not develop any TB coinfection. These incidences are quite high for transition from state 4 (CD4 cell count between 200 and 350) to state 5 (CD4 cell count below 200). For these states the immune system is still weak. As a result patients on antiretroviral drugs should consistently be screened for TB coinfection. Patients who had initially been diagnosed with TB before commencement of ART recover better from HIV/AIDS disease except that transitions to death for patients with CD4 cell count between 350 and 500 cells/mm^{3} are two times higher than that of patients who were not initially diagnosed with TB.
From this cohort, transitions to bad states are higher for males than for their female counterparts. This is quite pronounced on transitions from state 2 (CD4 cell count between 500 and 750) to state 3 (CD4 cell count between 350 and 500) where the hazards for males are 6 times that of females. This result is consistent with the findings from Maskew and others, they discovered that men gain fewer CD4 cell counts than did women [20]. An assessment of published studies by Castillo and others [21] from both resourcelimited and resourcerich countries suggest an improved survival outcomes for females than males. However, the studies they assessed do not show a clear sex disparity in the disease progression or in treatment effects of viral suppression and immunologic recovery.
The results from the fitted model show that the rates of immune recovery were much higher than the rates of immune deterioration which is an indication of effectiveness of treatment. Patients who started treatment when their CD4 baseline was at least 350 had higher rates of immune recovery than those who had a lower CD4 baseline. This result is commensurate with the findings from Moore and Keruly who also discovered that patients with baseline CD4 cell count above 350 cells/mm^{3} returned to nearly normal CD4 cell count after 6 years [22]. The probability of dying increases with decreasing CD4 count of the individual at enrolment. This is supported by the findings of [23–25], who also concluded that being in the AIDS defining stage leads to the highest probability of reaching the death state.
The mean sojourn times revealed that patients take longer time in the AIDS defining states (CD4 cell count below 200) before they move to the other states. Research has also shown that CD4 cell count rises gradually despite the suppressed viral load particularly in older patients. Hence, there is need to use both CD4 cell count and viral load in monitoring the efficacy of treatment. The younger people below the age of 40 have higher chances of immune recovery than the older ones. This finding is supported by some previous studies who concluded lower mean CD4 increases for older patients than younger patients [20, 26]. Alioum and others further argued that this could be caused by the fact that older subjects may have a reduced capacity to generate CD4 cells in response to the viral killing [10].
Although continuous time Markov models can handle multiple or recurrent outcomes compared to the Kaplan Meier analysis and Cox proportional hazards models, the assumption of constant hazard function that is frequently unrealistic [27] and puts limitations on the disease history behaviour [28], especially on HIV/AIDS progression for patients on ART. Some studies have shown that if a patient responds well to treatment and manages to achieve viral load suppression within the first 6 months, that patient is likely to continue responding well to treatment [29]. This goes against the Markov and memoryless properties of the models. Thus a limitation in the application of time homogeneous Markov processes.
Declarations
Acknowledgements
This study would not have been a success without the assistance of the Microbiology Department at the University of Venda in providing the secondary data through Professor Pascal O. Bessong.
Funding
Not applicable
Availability of data and materials
The data are available by contacting the corresponding author and can be submitted upon request.
Authors’ contributions
CS and DC drafted the manuscript. CS and DC contributed to the analysis and interpretation of the data. Both authors participated in critical revision of the manuscript drafts and approved the final version.
Authors’ information
CS is a PhD student and DC is a senior lecturer at the University of the Free State.
Ethics approval and consent to participate
Not applicable
Consent for publication
The authors give their consent to publish this work.
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Authors’ Affiliations
References
 Perelson AS, Nelson PW. Mathematical analysis of hiv1 dynamics in vivo. SIAM Rev. 1999;41(1):3–44.View ArticleGoogle Scholar
 Sonnenberg P, Glynn JR, Fielding K, Murray J, GodfreyFaussett P, Shearer S. How soon after infection with HIV does the risk of tuberculosis start to increase? A retrospective cohort study in south African gold miners. J Infect Dis. 2007;191:150–8.View ArticleGoogle Scholar
 Cingolani A, Lepri AC, Castagna A, Goletti D, De Luca A, Scarpellini P, Fanti I, Antinori A, d’Armino Monfarte A, Girardi E. Impaired CD4 Tcell count response to combined antiretroviral therapy in antiretroviralnaive HIVinfected patients presenting with tuberculosis as AIDSdefining condition. CID. 2012;54(6):853–61.View ArticleGoogle Scholar
 Ndumbi P, Falutz J, Pai NP, Tsoukas CM, et al. PLoS One. 2014;9(4):e94018. https://doi.org/10.1371/journal.pone.0094018.
 Phillips AN, Staszewski S, Weber R, Kirk O, Francidi P, Miller V, Vernazza P, Lundgren JD, Ledergerber B. HIV viral response to antiretroviral therapy according to the baseline CD4 cell count and viral load. JAMA. 2001;286:2560–7.View ArticlePubMedGoogle Scholar
 Abner EL, Nelson PT, Schmitt FA, Browning SR, Fardo DW, Wan L, Jicha GA, Cooper GE, Smith CD, CabanHolt AM, Van Eldik LJ, Kryscio RJ. Selfreported head injury and risk of latelife impairment and AD pathology in an AD center cohort. Dement Geriatr Cogn Disord. 2014;37:294–306.View ArticlePubMedGoogle Scholar
 Mullins LJ. Management and Organisational behaviour. 4th ed. London: Pitman Publishing; 1996.Google Scholar
 Foucher Y, Mathieu E, SaintPierre P, Durand JF, Daures JP. A semiMarkov based on generalised Weibull distribution with an illustration for HIV disease. Biom J. 2005;6:1–9.Google Scholar
 Longini IM, Clark WS, Byers RH, Ward J, Darrow WW. Statistical analysis of the stages of HIV infection using a Markov model. Stat Med. 1989;8:831–43.View ArticlePubMedGoogle Scholar
 Alioum A, Leroy V, Commenges D, et al. Effects of gender, age, transmission category, and antiretroviral therapy on the progression of human immuno virus infection using multistate Markov models. Group d'Epidemiologie clinique du SIDA en Aquitaine, Epidemiology. 1998;9(6):605–12.Google Scholar
 Reddy T. HIV disease progression in South Africa using multistate Markov models. SACEMA. 2011; http://www.sacemaquarterly.com.
 Binquet C, Le Teuff G, Abrahamovicz M, Mahboubi A, Yazdanpanah Y, Rey D, Rabaud C, Chirouze C, Berger JL, Faller JP, Chavanet P, Quantin C, Piroth L, Groupe InterCOrevih du NordEst (ICONE). Markov modelling of HIV infection evolution in the HAART era. Epidemiol Infect. 2009;137(9):1272–82. https://doi.org/10.1017/S0950268808001775. Epub 2009 Jan 12
 Grover G, Gadpayle KA, Swain PK, Deka B. A multistate Markov model based on CD4 cell count for HIV/AIDS patients on antiretroviral therapy (ART). Int J Stat Med Res. 2013;2:144–51.Google Scholar
 Gibson EL. Continuous time multistate models for survival analysis. WinstonSalem: Wake Forest University, Department of Mathematics; 2008.Google Scholar
 Halim D. Maximum likelihood estimation for generalized semiMarkov processes. Discrete event dynamics systems: theory and applications. Dep Ind Eng. 1996;6(2):73–104.Google Scholar
 Jackson CH. Multistate models for Panel Data: The msm package for R. J Stat Softw. 2013;38(8). http://www.jstatsoft.org/.
 Titman, A.C., Model diagnostics in multistate models of biological systems, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge, 2007.Google Scholar
 Kalbfleisch JD, Lawless JF. The analysis of panel data under a Markov assumption. J Am Stat Assoc. 1985;80:863–71.View ArticleGoogle Scholar
 Cox DR. The statistical analysis of dependencies in point processes. In Stochastic Point Processes. Wiley; 1972.Google Scholar
 Maskew M, Brennary AT, Westreich D, McNamara L, MacPhail P, and Fox MP. Gender Differences in Mortality and CD4 Count Response Among HIVPositive Patients Virally Suppressed Within 6 Months of Antiretroviral Therapy Initiation. J Womens Health. 2012; 22(00). https://doi.org/10.1089/jwh.2012.3585.
 Castilho JL, Melhekin VV, Sterling TR. Sex Differences in HIV Outcomes in the Highly Active Antiretroviral Therapy Era: A Systematic Review. AIDS Res Hum Retroviruses. 2014;30(5):446–56. https://doi.org/10.1089/aid.2013.0203.
 Moore RD and Keruly JC. CD4+Cell Count 6 Years after Commencement of Highly Active Antiretroviral Therapy in Persons with Sustained Virologic Suppression CID. 2007; 44: 441–6.Google Scholar
 Biadgilign S, Reda AA, Digaffe T. Predictors of mortality among HIV infected patients taking antiretroviral treatment in Ethiopia: A retrospective cohort study. AIDS Res Ther. 2012;9:15 https://doi.org/10.1186/17426405915.
 Goshu TA and Dessie ZG. Modeling progression of HIV/AIDS disease stages using semiMarkov processes. 2013.Google Scholar
 Lifson AR, Krantz EM. Grambsch PL. Macalino GE. CrumCianflone NF. Ganesan A, Okulicz JF, Eaton A, Powers JH, Eberly LE, Agan BK. Clinical, demographic and laboratory parameters at HAART initiation associated with decreased postHAART survival in a U.S. military cohort. AIDS Res Ther. 2012; 9(4).Google Scholar
 Hasibi M, Hajiabdolbaghi M, Hamzelou S, Sardashti S, Faroughi M, Jozani ZB, Alinaghi SAS. Impact of age on CD4 response to combination antiretroviral therapy: a study in Tahran, Iran. Orld J AIDS. 2014;4:156–62.View ArticleGoogle Scholar
 Lange JM, Hubbard RA, Inoue LYT, Minin VN. A joint model for multistate disease processes and random informative observation times, with applications to electronic medical records data. Biometrics. 2015;71(1):90–101.View ArticlePubMedGoogle Scholar
 SaintPierre P, Combescure C, Daures JP and Godard P. The analysis of Asthma control under a Markov assumption with use of covariates. Statistics in Medicine. 2003;22(24):3755–3770.Google Scholar
 Maartens G, Cotton M, Meintjes G, Mendelson M, Rabie H, Maharaj S. Clinical guidelines 9th Edition. Copyright Aids for AIDS management (Pty) Ltd. 2012.Google Scholar