Skip to main content

Understanding the Graphs

Individual audits have different graphical displays. The displays, and the information they provide for each individual audit, are described under the headings below and can also be found attached to each indivdual audit:

Cystectomy

Data quality

We have attempted to indicate the quality of data entry for cystectomy by comparing the returns with Hospital Episode Statistics (HES data) - this is information collected by every NHS hospital after a patient’s discharge from a surgical procedure. Unfortunately, HES data are not 100% reliable but ideally, all procedures recorded on HES will be entered by the surgeon on to the BAUS database for analysis. A coding percentage close to 100% suggests good recording; a percentage higher than this may indicate incorrect HES coding of procedures at the individual surgeon's hospital and a low percentage suggests incomplete recording by the surgeon or, again, poor HES coding.

Number of Operations

The number of operations indicates the number of radical cystectomy procedures undertaken by each surgeon. There is good evidence that better outcomes are obtained by surgeons who perform more procedures (high-volume surgeons), but this does not mean that low-volume surgeons have poor outcomes. These data include open, endoscopic, laparoscopic and robotic-assisted procedures for cancer of the bladder.

The data are taken from the BAUS Data and Audit System for procedures performed between 1 January 2014 and 31 December 2016.

The number of operations reported may be lower if a surgeon has only worked through part of the relevant time period.

This is the second year of publication for these data and we are currently working to identify the parameters necessary to risk stratify the data. 

Certain surgical techniques do appear to be associated with differing outcomes. For example the transfusion rates for open cystectomy appear to be higher in general when compared to a robotic assisted approach. This doesn't however mean that a person undergoing an open operation will have a poorer outcome. 

Transfusion rate

This is the proportion of patients (expressed as a percentage) requiring a blood transfusion following surgery.  It is often related to the complexity of the procedure and, separately to the underlying health of the patient.  It may also be an indicator of complications that have arisen following the procedure.

The transfusion rate reported does not take into account the patient’s pre-operative haemoglobin level.  If a patient is anaemic before the procedure, this may make him/her more likely to need a transfusion post-operatively. 

The reported tranfusion rate only identifies whether a patient received a blood transfusion during their admission for radical cystectomy; it does not identify whether the patient was re-admitted later for transfusion.  Patients requiring only a single unit of blood are recorded in the same way as those requiring multiple units of blood (i.e. they either did or did not receive a blood transfusion, with the number of units transfused not being recorded).

The transfusion rate reported may be lower if a surgeon operates only on less complex cases or on patients who are less sick; equally, it may be higher if the surgeon operates on more complex cases or patients.

The red vertical lines displayed on the graphs indicate an acceptable amount of variation (99% and 99.9% alerts). If the figures from your surgeon or unit are shown as lying within these inner boundaries (i.e. the dark blue bar is to the left of the red lines), then that surgeon/unit is graded as "acceptable".

We would look closely at surgeons/units if their figures lie beyond the 99% alert. If the blue bar is to the right of the 99.9% line, the surgeon is termed an “outlier”, and further investigation is essential to determine whether there may be a cause for concern.

Length of Stay

The length of stay following the operation can be influenced by the technique employed to carry out the procedure, an individual patient’s general health or complications of surgery, logistical issues (such as the distance they live from where the surgery is carried out) and individual surgeons' practice.

Average Patient Risk Profile

Some units/surgeons will be operating on patients with co-morbidities which put them at higher risk of complications, increased length of stay, transfusion, and ultimately post operative mortality. We are currently working to identify the parameters necessary to risk stratify the data. It is hoped that by presenting the data in this context, this will more accurately reflect an individual surgeons/units outcomes. Thus far we have identified, age >75, BMI, pre op anaemia, cardiopulmonary exercise tolerance & renal impairment as factors which increase risk.

N.B. BAUS has included all the data returned in our overall analysis but, when presenting individual surgeon’s results, we have excluded those surgeons who returned less than five cases for the year because any statistical analysis of such a low number would be invalid.

 

Nephrectomy

The nephrectomy graphs

There are four groups of information, displayed as graphs in the nephrectomy audit:

  1. The first group attempts to validate the BAUS data by comparing with Hospital Episode Statistics (HES data  ) - this is information collected by every NHS hospital after a patient’s discharge from a surgical procedure. Unfortunately, HES data are not 100% reliable. Ideally, all procedures recorded on HES will be entered by the surgeon on to the BAUS database for analysis. A coding percentage close to 100% suggests good recording; a percentage higher than this may indicate incorrect HES coding of procedures at the individual surgeon's hospital, whilst a low percentage suggests incomplete recording by the surgeon or, again, poor HES coding.
  2. The second group shows the number of times nephrectomy (removal of the kidney) has been performed by a hospital trust or surgeon over a year, so you can see if your surgeon carries out such surgery regularly or rarely. The graphs are sub-divided by the particular procedure, whether it was performed laparoscopically (including hand-assisted and robotic) or as an open operation, and whether it was performed for cancer or for a benign condition.
  3. The third group shows the risk-adjusted complications and transfusion rate, actual mortality and length of stay.
  4. The fourth, and final, group shows the average patient risk profile, alongside the patient profile risk for the individual surgeon or centre.

Risk adjustment – how and why it needs to be done

Patients vary because of their age, sex and the number of other illnesses they have (known as co-morbidities) as well as because of the extent and severity of their disease, e.g. large tumours involving adjacent organs. Some surgeons / centres may have more complex patients, others may have far fewer. This is known as patient casemix and needs to be taken into account when considering figures such as those shown, because high-risk patients are more likely to suffer complications. In addition, some procedures are inherently riskier than others. The combination of these factors is collectively known as “risk adjustment”.

How is the adjustment done?

The data have been analysed by statistical experts, using complex methods, to take into account many of these factors. The charts shown here display what the rate would have been had each surgeon/centre operated on the average case mix and are dependent on the number of procedures (sample size) actually reported per individual or centre; this is why they change from person to person.

The red bars indicate an acceptable amount of variation (99% and 99.9% alerts). If your surgeon / centre is shown as being within these inner boundaries (i.e. the dark blue bar is to the left of the red lines), then his or her practice is acceptable (see example below). We would look closely at those above the 99% alert. If the blue bar is to the right of the 99.9% line, the surgeon is termed an “outlier”, and further investigation is required to determine whether there may be a cause for concern.

Risk adjusted rate for transfusion

It is important to note that it is not always possible for risk adjustment to tell the whole story. Some surgeons are taking on extremely difficult, challenging surgery on very sick people, who may have relatively high morbidity rates. Others may be developing new techniques. These surgeons may, therefore, appear as outliers although they are the leading specialists in their field.

This dataset was originally set up in 2001 to evaluate the new technique of laparoscopic nephrectomy (keyhole removal of the kidney); it was never the intention to use these data for risk adjustment. Thus, the predictive accuracy (and the ability to risk-adjust every patient) is bound to be less than perfect. Small sample sizes reduce the precision of the risk adjustment and a difficulty with these data is that they are still in their early stages - data publication only began in 2013. We have, however, now added a number of additional questions to the dataset, to help improve this risk adjustment process for future years.

N.B. BAUS has included all the data returned in our overall analysis but, when presenting individual surgeon’s results, we have excluded those surgeons who returned less than five cases for the year because any statistical analysis of such a low number would be invalid.

More detailed information about the methods used for risk adjustment is available by clicking here ("How We Do Risk Analysis").

Average patient risk profile

Some risk factors (e.g. age, haemoglobin levels and other medical problems) can affect the outcome of nephrectomy in terms of complications, length of stay and the likelihood of requiring a transfusion.

The graphs show what percentage of the hospital or Consultant’s patients has each (potential) risk factor. This can indicate whether the hospital / surgeon operates on high-risk patients or, in fact, specialises in peforming complicated surgery such as cytoreductive procedures or major exploration of the vena cava.

 

PCNL

PCNL is the removal of stones from the kidney (or upper ureter) using a small puncture incision, into the kidney, in the skin of the affected side. Small instruments are then passed into the kidney (or upper ureter) to break up and remove the stones. PCNL can be performed with the patient lying either face down (prone) or face up (supine).

The headings below represent the three tables in the PCNL dataset:

Number of operations

The number of operations indicates how many PCNL procedures each surgeon has carried out. 

If a patient undergoes more than one PCNL procedure for the same stone, this has been recorded as two procedures.  If a patient has stones in both kidneys, operated on during the same anaesthetic, this has been recorded as two separate procedures.

The number of operations reported may be lower if a surgeon has only worked during part of the relevant time period.

Some surgeons may operate only on smaller, less complex stones, or on patients who are less sick, referring more complex stones and high-risk patients to other surgeons. As a result, the number of procedures that an individual surgeon performs, and their complexity, may be affected by these local referral pathways.

Transfusion rate

This is the proportion of patients (expressed as a percentage) requiring a blood transfusion following surgery.  It is often related to the complexity of the procedure and, separately to the underlying health of the patient.  It may also be an indicator of complications that have arisen following the procedure.

The transfusion rate reported does not take into account the patient’s pre-operative haemoglobin level, which may make the patient more likely to require a transfusion post-operatively. 

The reported rate only identifies whether a patient received a blood transfusion during their admission for PCNL; it does not identify whether the patient was subsequently re-admitted for a blood transfusion.  Patients requiring only a single unit of blood are recorded in the same way as those requiring multiple units of blood (i.e. they either did or did not receive a blood transfusion, with the number of units transfused not being recorded).

The transfusion rate reported may be lower if a surgeon operates only on smaller, less complex stones or on patients who are less sick; equally, it may be higher if the surgeon operates on more complex stones or patients.

Length of stay

The length of stay indicates the number of days a patient stayed in hospital following their procedure.  It is expressed as a median for each surgeon, with a mean and a range.  The length of stay is often related to the complexity of the procedure and, separately, to the underlying health of the patient.  It may also be an indicator of complications which have arisen following the procedure. 

The length of stay reported records only the number of days spent in hospital following the procedure.  It does not record subsequent nights spent if the patient is readmitted to hospital.

The length of stay reported may be shorter if a surgeon operates only on smaller, less complex stones or on patients who are less sick; equally, it may be longer if the surgeon operates on more complex stones or patients.

Average patient risk profile

Some risk factors (e.g. stone complexity, size and other medical problems) can affect the outcome of PCNL in terms of complications, length of stay and the likelihood of requiring a transfusion.

The graphs show what percentage of the hospital or Consultant’s patients has each (potential) risk factor. This can indicate whether the hospital / surgeon operates on high-risk patients or, in fact, specialises in peforming complicated surgery.

 

Radical Prostatectomy

Data quality

We have attempted to indicate the quality of data entry for prostatectomy by comparing the returns with Hospital Episode Statistics (HES data) - this is information collected by every NHS hospital after a patient’s discharge from a surgical procedure. Unfortunately, HES data are not 100% reliable but ideally, all procedures recorded on HES will be entered by the surgeon on to the BAUS database for analysis. A coding percentage close to 100% suggests good recording; a percentage higher than this may indicate incorrect HES coding of procedures at the individual surgeon's hospital and a low percentage suggests incomplete recording by the surgeon or, again, poor HES coding.

Number of operations and technique

The number of operations indicates the number of radical prostatectomy procedures undertaken by each surgeon. There is good evidence that better outcomes are obtained by surgeons who perform more procedures (high-volume surgeons), but this does not mean that low-volume surgeons have poorer outcomes. These data include open, endoscopic, laparoscopic and robotic-assisted procedures for cancer of the prostate.

The data are taken from the BAUS Data and Audit System for procedures performed between 1 January 2014 and 31 December 2014.

Interpretation of results

The number of operations reported may be lower if a surgeon has only worked through part of the relevant time period. This is the first year of publication for these data, so there is currently insufficient data for risk adjustment, but this may be possible with subsequent data sets.

Complication rate

The incidence of complications can vary from very minor (which do not alter the length of stay in hospital) to serious (which may require further intervention).  The data presented here indicate all reported post-operative complications labelled Clavien 3 and above (i.e. any complication requiring surgical, endoscopic or radiological intervention, which would prolong hospital stay).

Interpretation of results

These results need to be interpreted with caution because there are many factors that can affect outcome which are not immediately apparent from the raw data.  The accuracy of capturing and reporting complications can be variable, and surgeons who appear to have more complications may, in fact, be the ones who are most efficient at recording them.

The red bars displayed on the graphs indicate an acceptable amount of variation (99% and 99.9% alerts). If the figures from your surgeon or unit are shown as lying within these inner boundaries (i.e. the dark blue bar is to the left of the red lines), then that surgeon/unit is graded as "acceptable".

We would look closely at surgeons/units if their figures lie beyond the 99% alert. If the blue bar is to the right of the 99.9% line, the surgeon is termed an “outlier”, and further investigation is essential to determine whether there may be a cause for concern.

 

Stress Urinary Incontinence

Data quality

We have attempted to indicate the quality of data entry for stress urinary incontinence by comparing the returns with Hospital Episode Statistics (HES data) - this is information collected by every NHS hospital after a patient’s discharge from a surgical procedure. Unfortunately, HES data are not 100% reliable but ideally, all procedures recorded on HES will be entered by the surgeon on to the BAUS database for analysis. A coding percentage close to 100% suggests good recording; a percentage higher than this may indicate incorrect HES coding of procedures at the individual surgeon's hospital and a low percentage suggests incomplete recording by the surgeon or, again, poor HES coding.

Number of operations

The number of operations indicates the total number of procedures carried out by each surgeon for stress urinary incontinence. This includes tapes, colposuspension, periurethral bulking agents and autologous sling procedures. 

The number of operations reported may be lower if a surgeon has only worked through part of the relevant time period. It is important to note that Urologists undertake only a minority of the total number of primary procedures for this condition but probably do carry out a higher proportion of the procedures for recurrence.

The data represents approximately 70% of the total procedures for stress urinary incontinence carried out by Urologists in England in 2014.

Patient Reported Outcome Measures

1. ICIQ-UI Short Form Questionnaire

The ICIQ-UI Short Form is a patient reported outcome measure (PROM) which assesses the impact of symptoms of incontinence on quality of life and outcome of treatment.  Patients complete the form at the time of initial assessment or prior to surgery to report the level and impact of their symptoms of incontinence and then again at follow-up post-procedure at three months. 

The data presented here indicates the improvement in the patient’s perception of the change in severity of symptoms using the ICIQ-UI Short Form.  The ICIQ-UI Short Form is an international standard questionnaire used to assess urinary symptoms. 

The health score categories are recorded as % improved, unchanged or worsened.

2. Pad Use Per Day (PPD)

Pad use per day (PPD) is a method to quantify urinary incontinence.  The number of pads used per day is recorded, as stated by the patient, pre-operatively and post-operatively to indicate the severity of urinary incontinence.

Pad usage categories are recorded as % improved, unchanged or worsened in severity.

PPD usage is not reported for all patients who have undergone a procedure for stress urinary continence.

PPD is a method to quantify urinary incontinence but, as noted above, not all patients are asked to report on PPD usage, and not all patients who are actually do so.

N.B. PROMs are highly subjective measurements: there is, therefore, a significant chance of variations between patients in the assessment of symptoms and outcomes

Urethroplasty

Data quality

We have attempted to indicate the quality of data entry by:

  • validating the BAUS data by comparing with Hospital Episode Statistics (HES) data - this is collected by each NHS hospital after a patient’s discharge following a surgical procedure. Unfortunately, HES data is not 100% reliable for stricture surgery because procedures may be coded differently in different hospitals; we are currently investigating how to make this coding more accurate in the future. Ideally, all procedures recorded on HES will be entered by the surgeon on to the BAUS database for analysis. A coding percentage close to 100% suggests good recording; a percentage higher than this may indicate incorrect HES coding of procedures at the individual surgeon's hospital. A low percentage suggests incomplete recording by the surgeon or, again, poor HES coding.
  • measuring the interval between the procedure and data entry - a long delay may result influence the accuracy of recording of the details, and of any complications that arise both during surgery or in the 30 days after. Ideally, the interval between a urethroplasty and data entry should be less than 30 days.
  • measuring the percentage of patients for whom follow up data is recorded - ideally, at least 85% of patients should have completed follow-up assessment, to give a realistic picture of expected outcomes.

Scope of practice

This is an indication of the volume of urethral surgery carried out by an individual surgeon and unit, and is displayed as the number of cases carried out during the audit period. Higher numbers indicate a larger reconstructive practice.

Outcomes data for bulbar urethroplasty

The outcomes from bulbar urethroplasty can, at the present time, only be determined by looking at:

  • the length of stay following the operation - this can be influenced by an individual patient’s general health, logistical issues (such as the distance they live from where the surgery is carried out), and individual surgeons' practice,
  • the complication rates during (and in the 30 days after) surgery - complication recording may be influenced by the percentage of follow-ups performed, the interval between the operation and outpatient appointment, and data entry / data field completion. What appear to be high complication rates may be seen where there has been assiduous recording of all deviations from an expected clinical course.  At the time this data was collected the complications were not graded, although they are now. An apparently high complication rate may also occur when little follow-up data has been entered but that which has contains an isolated, or small number of, complications. Where high complication rates are reported further enquiries are made and explanations are sought from the surgeons concerned, these are reflected on the individual surgeon's outcome pages. 
  • the urine flow rate at outpatient review - the urine flow rate is an indication of the outcome from the surgery but may be influenced by bladder function, and by the volume of urine passed at the time of the test.