Progress: 18 July 2012

I'm getting some progress with my work and just happy to present these diagrams that seem to make the sense out of my work (more than my write up). After much deliberation on my research structure, I have decided this will by my methodology scheme:

Click to enlarge

Also I've finally made sense of the documents I'm using! I've tied it up with my discussion on UNDP. Basically it's going to be the documents on the programme level + UNDAF. Still trying to decide whether to include documents on the project level.

Click to enlarge

Research Structure

This picture from Krippendorff shows an idea of how I am going about my research method:

Click to enlarge

Although, this doesn't entirely capture what I want to do... Since I'm not sure I'm using content analysis alone. I think it is more like I'm building cases out of the documents; and I'm using content analysis + (something else, i.e. discourse analysis/semiotics/observations?) to make sense of the documents...

So the picture below maybe a better representation (my inputs in red)...

Click to enlarge

I plan to construct 15 mini (pseudo) cases to represent the risk management and outcome phenomena in each country; and I would then compare those similar phenomena to make an inference.

Does that sound sensible?

---

KRIPPENDORFF, K. 2004. Content analysis : an introduction to its methodology, Thousand Oaks, Calif., Sage.

Documents Thus Far

I'm streamlining my data... So far I have shortlisted my sources of documents to the following categoreis

Region: Europe and CIS
Timeframe: Programme cycles that fall between 2005 to 2011
Countries: Albania, Armenia, Azerbaijan, Bosnia and Herzegovina, Georgia,  Kazakhstan, Kyrgyzstan, Moldova, Romania, Serbia, Tajikistan, Turkmenistan, Ukraine, Uzbekistan and the former Yugoslav Republic of Macedonia.

Documents to be analysed:

1. UNDAF
2. CPD (past cycle and on-going cycle)
3. CPAP (past cycle and on-going cycle)
4. ADR (if any) - or - Country Programme Performance Summary (if without ADR)

Given the above list... I am looking potentially at 90 document (1 UNDAF, 2 CPDs, 2 CPAPS, 1 ADR/Summary for each country) already... I'm still deliberating if it will be necessary to look into the additional documents. Aside from the constraint of time and the sheer amount of volume of documents, I've listed my reservations about some of the documents.

1. Outcome Evaluations - Reservation(s): Not all country programmes have outcome evaluations for the specified period. And only selected outcomes are evaluated. Advantage(s): it would list the projects and outputs that contribute to achieving the outcomes.    
2. Project documents - Reservations(s): I'm still undecided whether I should look in depth into project documents since I'm looking at the programme level; maybe I could use it as a support. There is a risk that I would look at the thesis on the project level. Advantage(s): since the programme documents (so far) have not given me quality data for risk management, maybe the project documents could give me insights:  
3. Project evaluations - my reservations and advantages are the same as #2

I guess the question here really is:

Is the country programme documents sufficient to prove a point? If not then I would need to look into project documents.

---
Here are the sources for the documents

UNDP Europe and CIS Country Programme documents (UNDAF and CPD):
http://europeandcis.undp.org/news/show/8EEC429C-F203-1EE9-BB5E744EAA529CF8

ERC Website (UNDAF, Outcome and Project Reports):
http://erc.undp.org/

Assessment of Development Results Reports
http://web.undp.org/evaluation/country-evaluation.html

Also know that... not all country offices would have ADRs, but instead would have a Country Programme Performance Summary. I haven't found a repository... but through persistent search using the keywords "UNDP" and "Country Programme Performance" all the relevant summaries can be found.

Finding a Way Forward

After some correspondence with Eddie, Dennis and Alec... I think I have gathered my confidence that all is not lost! (Read about my panic attack here).

Right after the meeting with Dennis, it seemed clearer what I needed to do. I tried to organise the documents I have and what information I am going to gather from those documents to address my research objectives:

Late Night Library Musings

Been cracking my head on a research issue... Been here in the library all day, skimming through documents, coding, thinking and panicking. Did not even notice that time flew so fast! So night time falls and I decided to try draw the picture of my situation.

So this is a glimpse of how my brain looks like (I'm sure the jumbled thoughts in my head must be amplified by a 1000x). Pictures are a bit blurred but I need a form of reference of my musings. 

At this rate... I'm getting really worried. But I don't think I'll get anything more done for tonight... so off to home now...

--- 

Just in case you are interested in what I'm fussing about:

My research question is: To what extent does programme risk management practice contribute to achieving UNDP programme outcomes?

My research method is designed to be unobtrusive; thus I am heavily depending on using UN documents that have been already created and published. Using these documents  I am trying to infer the extent of (1) how UNDP country programmes uses/perceives  programme risk management and (2) how programme risk management is valued in the process of achieving programme outcomes.

Through my research so far I have been able to:

• Trim down my study's target population to 15 countries within the Europe and CIS region.
• Set the time-frame of the programme cycles to be within the years 2005-2011; the rationale being, so that the outcomes are already evident.
• Gather, complete and read the top 3 country programme documents (UNDAF, CPD and CPAP) of the 15 countries; where the documents have narrated how the programmes are supposed to be managed (but only some includes a risk management aspect)
• Skim through the ERC website, where the documents can give me inferences of the outcomes of the programmes.

What is causing me to worry is: so far I hardly see any explicit risk management aspect in the documents that I have. In addition, I've also gathered that the UN system has proposed and implemented an enterprise risk management framework; but I have realised that the framework was introduced in 2008, mid-way through the programme cycle (therefore will not be evident in the documents that I have).

I've struck gold!

Not literally... but I found this website which just gives me a huge relief!
http://europeandcis.undp.org/news/show/8EEC429C-F203-1EE9-BB5E744EAA529CF8

Although I  have done my own data mining for the past few weeks and have created my own database... this gives me the reassurance of the completeness of (or lack thereof) my data :)

Posting the link on my blog just to be sure I don't lose it.