Our University

Program Evaluation

Business Administration - Finance

Cost-Benefit Analysis

Cost-Effectiveness Analysis

Objective: to understand how CBA  and CEA could help in evaluating a program or project

Type of evaluations

•          . Goal-based evaluation

•          b. Process-based evaluation

•          c. Outcomes-based evaluation

•          Related to those type of evaluations:

•          Cost-Benefit Analysis CBA

•          Cost-effectiveness Analysis CEA

•          Financial Evaluation

Why CBA or CEA?

•          All programs aim to produce benefits that outweigh their costs.

•          Costs and benefits can be compared to determine the worthiness of a program.

•          Cost-benefit and cost-effectiveness analysis are the most common methods used to accomplish the comparison between costs and benefits.

•          Both analyses provide information about the net present value (NPV) of a program. NPV is an indicator of how much value an investment or project adds to the firm. The difference between the present value of cash inflows and the present value of cash outflows. NPV is used in capital budgeting to analyze the profitability of an investment or project. 

•          In CBA the benefits are transformed into monetary terms and compared to program costs.

•          In CEA benefits are transformed into non-monetary unit, such as lives saved, people attended, and are compared with program costs in dollars.

How do they work?

•          At the planning stage, CBA or CEA may be undertaken before the fact, based on estimates of anticipated cost and benefits.

•           They are tools analysis, particularly as ways to examine the net benefits of a proposed project or program involving large capital investments.

•          After a program has been in operation for some time, CBA and CEA may be used after the fact, to assess whether the actual costs of the program were justified by the actual benefits.

CBA

•          CBA is a quantitative analytical tool to aid decision-makers in the efficient allocation of resources.

•          CBA helps managers answer questions such as:

•          Does the proposal provide a net benefit to the community as a whole?

•          Should the proposed project, programme or policy be undertaken?

•          Should the project or programme be continued?

•          Which of various alternative projects or programmes should be undertaken?

•          The CBA could be done from three perspectives or points of view:

1. Individual’s perspective
2. government perspective
3. social perspective.

•          Definition of costs and benefits will usually differ from one perspective to the next.

The social perspective, for example accounts for all costs and benefits to society. However, the individual perspectives may help shed light on differing viewpoints about the worth of the program, or explain a program’s success or failure.

The individual’s perspective

•          It examines the program costs and benefits to the program participant: person, a family, a company or a non-profit organization.

•          CBA done from such a perspective often produce high benefit-cost ratios because the government or society subsidizes the program from which the participant benefits.

Government perspective

•          The analysis from a government perspective values costs and benefits from the point of view of the funding source.

•          It is basically a financial analysis, examining the financial costs and the direct financial benefits to the government.

•          Cash flows that would be examined: program administrative costs, direct cash outlays (expenditure), taxes paid, unemployment insurance, and others.

Ethics: Informed Consent from Program Participants

Ethics: Informed Consent from Program Participants

Note that if you plan to include in your evaluation, the focus and reporting on personal information about customers or clients participating in the evaluation, then you should first gain their consent to do so. They should understand what you're doing with them in the evaluation and how any information associated with them will be reported. You should clearly convey terms of confidentiality regarding access to evaluation results. They should have the right to participate or not. Have participants review and sign an informed consent form. See the sample informed-consent form.

How to Apply Certain Methods

Purposes and Formats of Questions Developing Questionnaires
Conducting Interviews
Conducting Focus Groups
Developing Case Studies

Selecting Which Methods to Use

Overall Goal in Selecting Methods:

The overall goal in selecting evaluation method(s) is to get the most useful information to key decision makers in the most cost-effective and realistic fashion. Consider the following questions:
1. What information is needed to make current decisions about a product or program?
2. Of this information, how much can be collected and analyzed in a low-cost and practical manner, e.g., using questionnaires, surveys and checklists?
3. How accurate will the information be (reference the above table for disadvantages of methods)?
4. Will the methods get all of the needed information?
5. What additional methods should and could be used if additional information is needed?
6. Will the information appear as credible to decision makers, e.g., to funders or top management?
7. Will the nature of the audience conform to the methods, e.g., will they fill out questionnaires carefully, engage in interviews or focus groups, let you examine their documentations, etc.?
8. Who can administer the methods now or is training required?
9. How can the information be analyzed?

Note that, ideally, the evaluator uses a combination of methods, for example, a questionnaire to quickly collect a great deal of information from a lot of people, and then interviews to get more in-depth information from certain respondents to the questionnaires. Perhaps case studies could then be used for more in-depth analysis of unique and notable cases, e.g., those who benefited or not from the program, those who quit the program, etc.

Four Levels of Evaluation:

There are four levels of evaluation information that can be gathered from clients, including getting their:
1. reactions and feelings (feelings are often poor indicators that your service made lasting impact)
2. learning (enhanced attitudes, perceptions or knowledge)
3. changes in skills (applied the learning to enhance behaviors)
4. effectiveness (improved performance because of enhanced behaviors)

Usually, the farther your evaluation information gets down the list, the more useful is your evaluation. Unfortunately, it is quite difficult to reliably get information about effectiveness. Still, information about learning and skills is quite useful.

Analyzing and Interpreting Information

Analyzing quantitative and qualitative data is often the topic of advanced research and evaluation methods. There are certain basics which can help to make sense of reams of data.

Always start with your evaluation goals:
When analyzing data (whether from questionnaires, interviews, focus groups, or whatever), always start from review of your evaluation goals, i.e., the reason you undertook the evaluation in the first place. This will help you organize your data and focus your analysis. For example, if you wanted to improve your program by identifying its strengths and weaknesses, you can organize data into program strengths, weaknesses and suggestions to improve the program. If you wanted to fully understand how your program works, you could organize data in the chronological order in which clients go through your program. If you are conducting an outcomes-based evaluation, you can categorize data according to the indicators for each outcome.

Planning Your Program Evaluation

Depends on What Information You Need to Make Your Decisions and On Your Resources.

Often, management wants to know everything about their products, services or programs. However, limited resources usually force managers to prioritize what they need to know to make current decisions.

Your program evaluation plans depend on what information you need to collect in order to make major decisions. Usually, management is faced with having to make major decisions due to decreased funding, ongoing complaints, unmet needs among customers and clients, the need to polish service delivery, etc. For example, do you want to know more about what is actually going on in your programs, whether your programs are meeting their goals, the impact of your programs on customers, etc? You may want other information or a combination of these. Ultimately, it's up to you.

But the more focused you are about what you want to examine by the evaluation, the more efficient you can be in your evaluation, the shorter the time it will take you and ultimately the less it will cost you (whether in your own time, the time of your employees and/or the time of a consultant).

There are trade offs, too, in the breadth and depth of information you get. The more breadth you want, usually the less depth you get (unless you have a great deal of resources to carry out the evaluation). On the other hand, if you want to examine a certain aspect of a program in great detail, you will likely not get as much information about other aspects of the program.

For those starting out in program evaluation or who have very limited resources, they can use various methods to get a good mix of breadth and depth of information. They can both understand more about certain areas of their programs and not go bankrupt doing so.

Key Considerations:

Consider the following key questions when designing a program evaluation.
1. For what purposes is the evaluation being done, i.e., what do you want to be able to decide as a result of the evaluation?
2. Who are the audiences for the information from the evaluation, e.g., customers, bankers, funders, board, management, staff, customers, clients, etc.
3. What kinds of information are needed to make the decision you need to make and/or enlighten your intended audiences, e.g., information to really understand the process of the product or program (its inputs, activities and outputs), the customers or clients who experience the product or program, strengths and weaknesses of the product or program, benefits to customers or clients (outcomes), how the product or program failed and why, etc.
4. From what sources should the information be collected, e.g., employees, customers, clients, groups of customers or clients and employees together, program documentation, etc.
5. How can that information be collected in a reasonable fashion, e.g., questionnaires, interviews, examining documentation, observing customers or employees, conducting focus groups among customers or employees, etc.
6. When is the information needed (so, by when must it be collected)?
7. What resources are available to collect the information?

Some Major Types of Program Evaluation

When designing your evaluation approach, it may be helpful to review the following three types of evaluations, which are rather common in organizations. Note that you should not design your evaluation approach simply by choosing which of the following three types you will use -- you should design your evaluation approach by carefully addressing the above key considerations.

Goals-Based Evaluation

Often programs are established to meet one or more specific goals. These goals are often described in the original program plans.

Goal-based evaluations are evaluating the extent to which programs are meeting predetermined goals or objectives. Questions to ask yourself when designing an evaluation to see if you reached your goals, are:
1. How were the program goals (and objectives, is applicable) established? Was the process effective?
2. What is the status of the program's progress toward achieving the goals?
3. Will the goals be achieved according to the timelines specified in the program implementation or operations plan? If not, then why?
4. Do personnel have adequate resources (money, equipment, facilities, training, etc.) to achieve the goals?
5. How should priorities be changed to put more focus on achieving the goals? (Depending on the context, this question might be viewed as a program management decision, more than an evaluation question.)
6. How should timelines be changed (be careful about making these changes - know why efforts are behind schedule before timelines are changed)?
7. How should goals be changed (be careful about making these changes - know why efforts are not achieving the goals before changing the goals)? Should any goals be added or removed? Why?
8. How should goals be established in the future?

Process-Based Evaluations

Process-based evaluations are geared to fully understanding how a program works -- how does it produce that results that it does. These evaluations are useful if programs are long-standing and have changed over the years, employees or customers report a large number of complaints about the program, there appear to be large inefficiencies in delivering program services and they are also useful for accurately portraying to outside parties how a program truly operates (e.g., for replication elsewhere).

There are numerous questions that might be addressed in a process evaluation. These questions can be selected by carefully considering what is important to know about the program. Examples of questions to ask yourself when designing an evaluation to understand and/or closely examine the processes in your programs, are:
1. On what basis do employees and/or the customers decide that products or services are needed?
2. What is required of employees in order to deliver the product or services?
3. How are employees trained about how to deliver the product or services?
4. How do customers or clients come into the program?
5. What is required of customers or client?
6. How do employees select which products or services will be provided to the customer or client?
7. What is the general process that customers or clients go through with the product or program?
8. What do customers or clients consider to be strengths of the program?
9. What do staff consider to be strengths of the product or program?
10. What typical complaints are heard from employees and/or customers?
11. What do employees and/or customers recommend to improve the product or program?
12. On what basis do employees and/or the customer decide that the product or services are no longer needed?

What is Evaluation?

What is Evaluation?

So What is Program Evaluation?

First, we'll consider "what is a program?" Typically, organizations work from their mission to identify several overall goals which must be reached to accomplish their mission. In nonprofits, each of these goals often becomes a program. Nonprofit programs are organized methods to provide certain related services to constituents, e.g., clients, customers, patients, etc. Programs must be evaluated to decide if the programs are indeed useful to constituents. In a for-profit, a program is often a one-time effort to produce a new product or line of       products.

Some Myths About Program Evaluation

1.. Many people believe evaluation is a useless activity that generates lots of boring data with useless conclusions. This was a problem with evaluations in the past when program evaluation methods were chosen largely on the basis of achieving complete scientific accuracy, reliability and validity. This approach often generated extensive data from which very carefully chosen conclusions were drawn. Generalizations and recommendations were avoided. As a result, evaluation reports tended to reiterate the obvious and left program administrators disappointed and skeptical about the value of evaluation in general. More recently (especially as a result of Michael Patton's development of utilization-focused evaluation), evaluation has focused on utility, relevance and practicality at least as much as scientific validity.

2. Many people believe that evaluation is about proving the success or failure of a program. This myth assumes that success is implementing the perfect program and never having to hear from employees, customers or clients again -- the program will now run itself perfectly. This doesn't happen in real life. Success is remaining open to continuing feedback and adjusting the program accordingly. Evaluation gives you this continuing feedback.

3. Many believe that evaluation is a highly unique and complex process that occurs at a certain time in a certain way, and almost always includes the use of outside experts. Many people believe they must completely understand terms such as validity and reliability. They don't have to. They do have to consider what information they need in order to make current decisions about program issues or needs. And they have to be willing to commit to understanding what is really going on. Note that many people regularly undertake some nature of program evaluation -- they just don't do it in a formal fashion so they don't get the most out of their efforts or they make conclusions that are inaccurate (some evaluators would disagree that this is program evaluation if not done methodically). Consequently, they miss precious opportunities to make more of difference for their customer and clients, or to get a bigger bang for their buck.

What is Business Administration?

What is Business Administration?

The concept of business administration encompasses all aspects of owning and operating a business. This includes everything from business management to marketing of the goods and services being offered. Universities and colleges offer bachelor’s degrees in business administration and many also offer masters degrees and PHDs. These are certainly helpful, particularly if someone is planning to try and find work with a large corporation, but not absolutely necessary for those individuals starting a small business on their own.

The concept of business administration encompasses all aspects of owning and operating a business. This includes everything from business management to marketing of the goods and services being offered. Universities and colleges offer bachelor’s degrees in business administration and many also offer masters degrees and PHDs. These are certainly helpful, particularly if someone is planning to try and find work with a large corporation, but not absolutely necessary for those individuals starting a small business on their own.

What is business administration?

The study of business administration focuses on developing analytical, decision-making and problem solving skills.  It also looks at communication skills and styles as well as interpersonal skills.  Business Administration develops an appreciation for how businesses plan strategically, the role of leadership, and management of people, resources and information.

What degree do I study for business administration?

The Bachelor of Business Administration (BBA) degree is designed to meet the demands of employers by developing students' leadership skills, their ability to think strategically, and to effectively manage organisational resources.  The BBA also aims to produce graduates with good people management skills.

The BBA structure has a certain amount of flexibility and it allows you to fit in one of the majors from the Commerce degree such as accounting, finance, marketing, international business or corporate sustainability.

Is there professional body accreditation?

Professional body accreditation is not required for practising in the profession. 

What careers are there in business administration?

The BBA is designed to produce ‘all-rounders’.  Graduates will have technical knowledge combined with leadership and people management skills.  The range of careers that we expect our BBA graduates to achieve include:

• Management Consultant
• Business Development Manager
• Human Resource Manager
• Marketing Manager
• Corporate Services
• Business Operations
• Public Administration

The concept of business administration encompasses all aspects of owning and operating a business. This includes everything from business management to marketing of the goods and services being offered. Universities and colleges offer bachelor’s degrees in business administration and many also offer masters degrees and PHDs. These are certainly helpful, particularly if someone is planning to try and find work with a large corporation, but not absolutely necessary for those individuals starting a small business on their own.

What is business administration?

The study of business administration focuses on developing analytical, decision-making and problem solving skills.  It also looks at communication skills and styles as well as interpersonal skills.  Business Administration develops an appreciation for how businesses plan strategically, the role of leadership, and management of people, resources and information.

What degree do I study for business administration?

The Bachelor of Business Administration (BBA) degree is designed to meet the demands of employers by developing students' leadership skills, their ability to think strategically, and to effectively manage organisational resources.  The BBA also aims to produce graduates with good people management skills.

The BBA structure has a certain amount of flexibility and it allows you to fit in one of the majors from the Commerce degree such as accounting, finance, marketing, international business or corporate sustainability.

Is there professional body accreditation?

Professional body accreditation is not required for practising in the profession. 

What careers are there in business administration?

The BBA is designed to produce ‘all-rounders’.  Graduates will have technical knowledge combined with leadership and people management skills.  The range of careers that we expect our BBA graduates to achieve include:

• Management Consultant
• Business Development Manager
• Human Resource Manager
• Marketing Manager
• Corporate Services
• Business Operations
• Public Administration

Do I have the right personal attributes?

Professionals working in Business Administration need to have:

• good people skills
• analytical skills
• high standard of oral and written communication
• logical and lateral problem solving
• ability to think critically and strategically