Brigham Young University Brigham Young University
BYU ScholarsArchive BYU ScholarsArchive
Instructional Psychology and Technology
Graduate Student Projects
Instructional Psychology and Technology
2023-06-13
But What If It Was a Cartoon? Animated Instructional Videos for But What If It Was a Cartoon? Animated Instructional Videos for
the California Debt and Investment Advisory Commission the California Debt and Investment Advisory Commission
Faith Jennings
Follow this and additional works at: https://scholarsarchive.byu.edu/ipt_projects
BYU ScholarsArchive Citation BYU ScholarsArchive Citation
Jennings, F. (2023). But What If It Was a Cartoon? Animated Instructional Videos for the California Debt
and Investment Advisory Commission. Unpublished masters project manuscript, Department of
Instructional Psychology and Technology, Brigham Young University, Provo, Utah. Retrieved from
https://scholarsarchive.byu.edu/ipt_projects/64
This Design/Development Project is brought to you for free and open access by the Instructional Psychology and
Technology at BYU ScholarsArchive. It has been accepted for inclusion in Instructional Psychology and Technology
Graduate Student Projects by an authorized administrator of BYU ScholarsArchive. For more information, please
contact [email protected].
1
But What If It Was a Cartoon?
Animated Instructional Videos
for the California Debt and Investment Advisory Commission
Faith Jennings
Design Project Report
Masters
Instructional Psychology & Technology, Brigham Young University
2
Purpose
The purpose of this project was to design Module 6 of an eight-video module curriculum for the
California Debt and Investment Advisory Commission (CDIAC). Elected officials from across
California are the learners for the module.
Background
California is the most populous state in the United States. It also has the largest economy of
any state in the U.S.; in 2021 California’s gross state product was $3.4 trillion. If California’s
economy is considered on its own it would be the fifth largest in the world and larger than
those of India, the United Kingdom, or France (“Economy of California,” 2023). Managing public
investment and debt is key to how California is governed and its economy functions. The
California Debt and Investment Advisory Commission (CDIAC) is the state’s clearinghouse for
public debt issuance information. Created in 1981 by state legislation, CDIAC is charged with
“providing information, education, and technical assistance on debt issuance and public fund
investments to local public agencies and other public finance professionals” (About CDIAC).
CDIAC’s activities are “classified into three general program areas: data collection and analysis,
policy research, and education” (About CDIAC).
This project concerns CDIAC’s educational mission. The guidelines and regulations surrounding
public debt can be difficult to parse, even for public finance professionals. Newly elected
officials often have a larger knowledge gap about how public debt functions and how to
navigate the systems already in place in order to achieve their goals. In recognition of this,
CDIAC has created an education portal with a distinct education path for elected officials (along
with paths for municipal debt issuance education and public fund investment education).
The first training resource CDIAC created for elected officials was an expansive PDF document
outlining definitions and explanations of different municipal finance terms and concepts. It was
encyclopedic in format. CDIAC posted this PDF encyclopedia on their website for officials and
other learners to download and access. This PDF contained an expansive range of information,
but they found that its format as one large document was difficult for learners to navigate.
During 2019 and 2020 CDIAC reformatted this encyclopedia as a WordPress website. Having
the encyclopedia in this new online, searchable format made it easier for learners to access the
information they needed. However, for learners to search for specific information they first
needed to know what specific information they were looking for. This was a high barrier for
many learners. Many newly elected officials did not have robust mental models about how
municipal finance functioned; therefore, it was difficult for learners to absorb new information.
As an experiment the CDIAC steering committee decided to create a module explaining finance
concepts to help learners learn in a systematic way. CDIAC worked with a local community
college to create a pilot video to gauge stakeholder interest in an online, video-based learning
module. Stakeholders decided that animated videos would be a more effective form of learning
and CDIAC solicited proposals from vendors to produce 8 instructional video modules under the
Elect>Ed identity to instruct elected officials on different aspects of public debt.
3
At this point Advanced Problem Solving (APS) was brought on as a partner. APS took the video
the community college had worked on and built it out into a full module which was released in
2019. Immediately after its release APS began working on additional modules with eight total
modules being planned. The purpose of this project is to produce the videos for a combined
Module 6/7,
Marketing and Structuring Strategies for a Win-Win Debt Issuance
.
Project Needs and Constraints
Audience Needs
The audience for CDIAC Elect>Ed training is made up of elected officials at all levels of
California government (school board, city, state agencies, etc.) These adult individuals vary in
age, educational and socioeconomic background, familiarity with technology, and experience
making financial decisions. As explained by Kerrie Dunn, director of instructional design at APS
in personal communication, these elected officials work with credentialed staff members who
are already knowledgeable about municipal finance. While these staff members prepare
documents and propose financial approaches to meet agency objectives, the “elected officials
should have (or gain) a strong understanding of what it means to be a fiduciary as an elected
official and should gain a basic understanding of the considerations and possible consequences
related to making/approving financial decisions” (K. Dunn, personal communication, February
22, 2023).
Most of these learners do not have expertise in public finance, so is critical to the success of the
module that the financial concepts are taught clearly in a mode that is accessible for beginners.
These officials are responsible for committing their constituents to debt structures that can
extend 30 years or farther into the future which will have long-lasting consequences. The
financial professionals at CDIAC saw a need to create this curriculum to help elected officials
understand the intricacies of municipal debt so these officials will be empowered to structure
the bonds their agencies issue in the most beneficial way for their constituents.
While many of the officials are not well-versed in municipal finance, many of them are quite
successful in their main careers. They are capable adults who were elected or appointed to
positions of authority and it is important not to patronize them.
Environmental Analysis
Stakeholders
The client for this training is the California Debt and Investment Advisory Commission (CDIAC)
and its staff. The commission consists of nine members: the State Treasurer, the Governor or
the Director of Finance, the State Controller, two local government finance officials, two
Assembly Members, and two Senators and while they are stakeholders in this training it is their
staff members that will serve as Subject Matter Experts and be directly involved in the
development of this module. The CDIAC SMEs are Robert Berry, Angelica Hernandez, and Tara
Dunn.
4
Product Environment
The mandate from the client to APS is “to build the Elect>Ed courses so that they are accessed
via an LMS, but responsive on both desktop and mobile devices” (Dunn, 2023). Besides creating
the module, APS also manages the LMS. Most of the learners enroll in the LMS using their
professional email accounts but it is unknown if they are taking the courses at work or at home.
According to Kerrie, “That line is pretty blurry for elected officials” (2023).
Considering both instructional theory and the environmental factors, it will be important to
divide the videos into more-easily absorbed chunks. This will aid in maintaining learner
engagement and make it easier for learners to pause and then resume the module if they have
to step away and continue their learning in multiple sessions.
Precedent
Many aspects of this project were influenced by the five preceding modules in the curriculum
(see Figure 1): the format of the module, the types of assessment items used, the animation
style, character and icon art, etc… At times it was challenging to fit my design within these
confines.
Figure 1: a view of preceding modules in the curriculum.
Budget
The budget for this project is proprietary and as such, I am unable to include some of the
details in this report. However, after reviewing the different factors involved, it is my view that
the only significant budgetary constraint that existed involved the contract animator. APS’s
contract with the animator included three rounds of revisions. If there were excessive revisions
5
beyond those, APS would have been charged for the extra updates. As a designer, this was
definitely a concern. Fortunately, we took care of all necessary revisions without the allowed
three rounds.
Addressing this concern influenced my strategy for dealing with client feedback. After
reviewing each draft of the video with the clients I made a concerted effort to ensure that I
understood their requested changes. I usually created new storyboards based on the client
feedback, sent the storyboard to the client for their approval, and then sent the new storyboard
onto the animator only after receiving confirmation that I had captured the client’s intent. I
also made an effort to bundle as many changes together as possible so there would be fewer
rounds of revision and I think this was a successful strategy.
According to Zippia.com, the average hourly rate for an animator in Utah is $41.00. Because
this particular animator has worked on several of the CDIAC modules previously, I am
estimating that his hourly rate is closer to $46 an hour. I am also estimating that it will take 30
hours to create drafts of each of the parts of the module–60 hours total. Using these
parameters, I am estimating the budget for animation to be approximately $3,000 once
revisions are added.
Developing prototypes for the animator, which was my job, was another major cost. I spent
approximately 100 hours researching the prior modules, developing the storyboards, and
meeting with clients. At a rate of $20 an hour this is a total of $2,000. In addition to this, I am
estimating that it took 3 hours of production work by my more-experienced team members to
get the versions of the module loaded into the LMS for review and for release at a rate of $33 a
hour (The 75
th
percentile rate listed for instructional designer on Zippia.com) which totaled is
$99.
The last remaining cost was hiring a professional voice artist to record the narration for the
module. APS typically contracts for “full service” voice over narration. “Full service” means
the talent has their own sound studio and equipment, does all the recording and editing, and
then sends APS the clean final files. APS has used the same narrator for all of the CDIAC
modules. For Module 6 they paid her $1,000 for full service narration of a 30-minute recording,
with a turn-around time of 24-48 hours from when she was sent the script. This price also
included the possibility of 2 rounds of edits with a 48-hour turnaround time frame.
In total, the estimated active cost to produce this module was $5,859. This is $1,499 higher
than I estimated in my proposal. This is because my development work took me 20 hours
longer than anticipated, and I did not include the three hours of LMS work to releasing the
module or the cost of narration in my proposal estimate.
Timeline
The actual project timeline took six weeks longer than I originally anticipated. Much of this
delay was the result of several stages lasting a few days longer than I had planned. While I
attempted to work in an agile manner, focusing on producing minimal viable products to keep
my iterations quick, the development process had certain waterfall stages checkpoints built in.
For example, we needed to receive the draft videos back from the animator in order to review
6
them. We had one notable delay: the animator we used experienced health problems during
February and March 2023 which added a few weeks to the timeline. Fortunately, he made a
full recovery.
October / November 2022
• Review existing Elect>Ed modules
• Script Review
• Receive the script from CDIAC and
review it for clarity.
• Send suggested edits to CDIAC for
their acceptance/rejection.
• Text of script is finalized
December 2022 / January 2023
• Begin development of storyboard.
• Refer to pre-existing visual glossary
and use existing art for previously
mentioned concepts.
• Use PowerPoint create slides for
animated scenes
• Meet weekly with SMEs to review in-
progress work as the storyboard is
in development.
February 2023
• Send script to voice over artist to
record narration
• Continue meeting weekly with SMEs
to revise iterations of storyboard
• Hand off Part 1 of storyboard along
with narration recording to animator
• Continue meeting weekly with SMEs
to revise Part 2 of storyboard
March 2023
• Animator creates draft video of Part
1 with SMEs
• Review Part 1 with SMEs
• Create new storyboards for Part 1 as
needed for revisions requested by
SMEs
• Hand off Part 2 storyboard to
animator
April 2023
• Animator creates draft video of Part
2
• Review Part 2 with SMEs
• Create new storyboards for Part 2 as
needed for revisions requested by
SMEs
• Coordinate any needed changes to
videos of Part 1 and Part 2
• Review assessment items provided
by clients
• Suggest updates to assessment
items as needed
May 2023
• Assemble videos and assessment
questions into draft module
• Module is reviewed by myself, APS,
CDIAC SMEs, and CDIAC steering
committee
• Any needed revisions are made
• Module is added to CDIAC education
portal and goes live
• An email to registered elected
officials announcing the new module
is sent through the LMS
Content or Task Analysis
This project is the sixth module in a series of eight modules. Due to business considerations
APS develops the modules using scripts provided by the client without conducting any formal
content analyses. I did not have the ability to alter the established development process, but for
the purposes of this project I have conducted a content analysis of the module.
This module,
Marketing and Structuring Strategies for a Win-Win Debt Issuance
, coves abstract
financial concepts and strategies. Most of the instruction is based on defining these concepts
using facts or outlining rule/principles to explaining when specific strategies may be
2
advantageous for elected officials to use. Items in the content analysis below are categorized
according to the key below.
Task Analysis Key
•
•
•
•
3
•
•
•
•
•
•
•
•
•
•
•
•
4
•
•
•
•
•
•
•
•
•
5
•
•
•
•
•
•
•
•
•
•
all
6
•
•
•
•
•
•
•
•
•
•
•
•
7
•
•
•
•
•
•
•
•
•
•
8
•
•
•
•
•
•
•
•
•
o
•
•
•
•
9
•
o
•
•
•
•
•
•
•
10
•
•
•
•
•
Content Analysis Results
After conducting this content analysis, the early encyclopedic roots of the project are especially
evident. The knowledge gap between the SMEs and the learners is large, and the strategy to
bridge it included in the client-provided script involves fitting as many topics into the module as
possible. Most of the module is spent explaining concepts by sharing relevant facts or outlining
required procedures. The module also includes explanations of the relationships between
multiple concepts such as the higher the risk is for investors, the higher the agency’s borrowing
costs.
Product Design
My experience with this project has taught me valuable lessons about completing the
instructional design process while simultaneously navigating the realities of being an employee
(and a junior employee at that). Leaning on the instructional design processes I learned during
my program, I had ideas about ways the module could be structured to improve learning
outcomes.
11
The previous modules consist of approximately 10-minute videos followed by brief multiple-
choice knowledge checks. When I evaluated this structure considering Merrill’s First Principles of
Instruction, it most closely resembles the “Tell-Ask” structure that Merrill finds to be “the least
effective instructional strategy” (Merrill, 2018, p. 267). However, evaluating this product against
Merrill’s First Principles helped me understand which specific deficits I wanted to address.
Activation
The learners will each approach the module with their own unique model of prior knowledge.
To help learners activate their knowledge as a foundation for the new topics that are taught in
this module, I would have liked to display a brief message before the module begins
encouraging the learners to review previous modules in the curriculum before completing this
module. Because this is the sixth module out of a total of eight, it was not feasible to make this
change in format for business reasons.
Demonstration
Since the targeted learners serve as elected officials in a variety of positions, the modules have
avoided using specific examples that would further demonstrate the principles taught. The
content is trying to walk the line of being specific enough to be relevant to all of the elected
officials participating in the module while being general enough not to alienate any of the
officials. Merrill says “Much instruction consists largely or entirely of presentation. What is often
missing is demonstration” (Merrill, 2018, p. 267). It is this lack of demonstration that I believe
ultimately limits how deep the learning from the module can be.
Application
The format of the completed module does not provide learners opportunities to apply what they
are learning to real-life scenarios. As Hill and Jordan say, “relevance helps the student to want
to learn the material by helping them understand how the material relates to their needs or
how it can relate to improving their future” (2021, pg. 128). When dealing with an abstract and
consequential topic like municipal finance, highlighting the relevance to the learners is
important to keep learners engaged.
Integration
Merrill discusses integration as being part of deep learning which requires learners to integrate
their newly acquired skills into their existing mental modes and he suggests one way of doing
this is having learners “share, reflect on, and defend their work by peer-collaboration and peer-
critique” (2018, pg. 268). Considering the module with this mind, I believe it would improve the
learning experience if after viewing the videos the elected officials could gather annually for a
workshop and presented with problem-based scenarios that they worked on as groups. Ideally,
officials would be grouped together with those in similar roles: school district officials, county
officials, city officials, etc…. Each group would be presented with a unique scenario similar to
ones they may encounter in their roles as officials. They would then use the knowledge gained
from viewing the module to craft a solution, after which each group would present their findings
about and receive peer feedback. As Merrill says, “Learners need to apply their newly acquired
skills to actually doing a task or actually solving a problem” (2018, pg. 268).
12
Problem-Centered
In previous modules the knowledge check questions focused on definitions and recall. While I
believe it would be optimal to incorporate a problem-based activity/workshop into the module,
it is not feasible within the confines of this project. One alternative would have been to
incorporate situational, problem-based questions into the module knowledge checks.
However, with my project, the client/SMEs provided the assessment questions themselves. I
had the opportunity to make suggestions about the specifics of the questions, which I did, but
not to drastically change the format of the questions. The eight assessment items in the
module all fall into the “Understand” or “Remember” categories of Bloom’s Taxonomy (see
Figure 2 below). Using only these types of does not take advantage of the opportunity to place
information in relevant contexts for the learners. I would have liked to have added one
scenario-based item to the module, but I did not have the ability to make this change to the
pre-determined format.
Figure 2: Blooms Taxonomy, Vanderbilt University Center for Teaching, CC BY 2.0
Improvements I Made
For business reasons I was unable to significantly alter the format of the assessment items.
However, I was able to reduce the cognitive load on learners by altering the structure of the
module. Previously, modules in this series have been formatted as two or three video segments
each followed by a few assessment questions. The module I worked on was especially lengthy,
with two videos clocking in at 29 minutes total. Each video was in the range of 13-15 minutes
long. Research has shown that learner engagement begins to decline when instructional videos
are longer than about 6 minutes (Brame, 2017). Considering this guideline, I advocated for the
idea of dividing the two videos into smaller chunks, each followed by the single assessment
question that related to the preceding material (See Figure 3).
13
Figure 3: original and revised module formats.
My supervisor was supportive of this suggestion. The clients had some reservations, mainly
that the format of this module would be different from the five other modules that preceded it
and that doing so would interrupt the narrative flow that they had written into the script. After
discussion of how this module was particularly abstract and citing the research that shows
learners engage better with shorter videos, the clients agreed to structure this module in this
format.
Design Details
I began this project by reviewing the previous five modules released in the series which deal
with other aspects of municipal finance for elected officials. Next, I reviewed the client-
provided script for Module 6. Knowing that the client was eager to cut word count where
possible (the word count being specified in the contact they had with my employer, APS) I
suggested edits where possible, sometimes rewriting sentences to use a more active voice,
sometimes simplifying sentences for increased clarity.
Creating the storyboard that translated the script into visuals for the motion artist was the bulk
of my design work. With such abstract subject matter, I took care to design the visuals to
complement the audio. Research has shown that using auditory instruction along with visuals
can improve learning outcomes (Clark, R., Nguyen, F., Swell, J., 2005).
The Product
The product is video module, hosted on CDIAC’s education portal. Below is a screenshot of the
module inside CDIAC’s LMS.
14
Figure 4: a view of Module 6 inside the CDIAC portal
After revising the format, the module is structured as outlined below.
Module Element
Description
Video 1 (3:30)
Agency objectives, municipal bond market, types of investors
Assessment Item 1
Drag-and-drop item re: investor segments
Video 2 (1:22)
Risk and the yield curve
Assessment Item 2
Multiple response re: the yield curve
Video 3 (3:57)
Competitive, negotiated, and private Placement sales methods
Assessment Item 3
Multiple response re: sales methods
Video 4 (3:13)
Labeled debt, TIC, & AIC
Assessment Item 4
Multiple response re: TIC and AIC
Video 5 (1:56)
Default risk and revenue sources
Assessment Item 5
Drag-and-drop re: spectrum of credit risk
15
Video 6 (4:09)
Size of issue, Serial/Term bonds
Assessment Item 6
Multiple response re: serial and term bonds
Video 7 (6:23)
Discount/Premium bonds, taxable/tax-exempt bonds, variable rate bonds,
CABs, optional redemption
Assessment Item 7
Drag-and-drop re: bond structures
Video 8 (4:56)
Refunding bonds, credit rating, agency objectives vs. investor preferences
Assessment Item 8
Multiple response re: credit rating
Figure 5: outline of video segments and assessment items that make up the module.
Link to Product
The full module, Module 6: Marketing and Structuring Strategies for a Win-Win Debt Issuance, is
hosted on the CDIAC education portal which requires a free registration to enter.
I have published a review link to the module. If it is helpful, you may leave feedback directly in
the module.
Video Walkthrough
https://www.loom.com/share/9088a163dc724396aca2d336c91010e6
Design Process and Evolution
Researching Precedent Products
I was excited, but intimidated, to come on board this project because the module I was
designing was the sixth in a series of eight planned modules and there was a lot of preceding
material to become familiar with. I spent a couple of weeks reviewing the previously-released
modules to research the content they contained. In addition, I researched the ways in which
that content was depicted visually, taking note of the icons used and the ways different
concepts were depicted. Over the course of the first five modules a visual style and glossary of
characters and icons had been developed, so it was important that the storyboards I created for
Module 6 utilize the relevant characters and icons where appropriate to be consistent with the
other modules.
Prototyping
The main chunk of my design process was spent creating prototype iterations. For my
storyboards I took the client-provided script and cut it into sections which I pasted onto
PowerPoint slides. Next, I created storyboards by inserting content or existing characters and
icons onto the slides (see Figure 6). As part of the storyboards I included motion instructions in
the Notes field of the slides for the animator (see Figure 7).
16
Figure 6 A draft of the storyboard I created in PowerPoint for Part 1 of the module.
17
Figure 7 Animation notes included in the storyboard for the motional graphic artist.
During the prototyping phase my supervisor, another instructional designer, and I met weekly
with the clients/SMEs to review the storyboards and gather their feedback. Sometimes I had to
conduct my own research outside of these meetings to better understand the financial concepts
I needed to teach visually. After every meeting I revised the storyboards in consideration of
their feedback. If I was stuck on a scene, I was able to get feedback from my supervisor or
team member which was helpful. Because the content was so abstract, some concepts
required multiple treatments before they were finalized.
The clients provided assessment questions they had written for the module. The APS team
reviewed the questions and suggested some format changes, but largely left the content as
written by the client. This was a business decision: one made in light of the history of the
preceding modules, the client/vendor relationship, and the current module’s deadline. I would
have liked to have included at least one problem-based question in the assessment, but it was
not possible because of the circumstances mentioned above.
However, we were successful in making one significant change to the format. My supervisor
and I advocated with the client to break the two videos up into eight chunks each followed by
its relevant assessment question to improve the cognitive load of the module. After discussion
18
(some of which was a little awkward) the clients agreed to try the format change. I consider
this a major win.
Production
Once the clients signed off on our storyboards, my team sent them off to the motion graphic
artist, or animator. We also sent the script to a voice artist who recorded the script as a
narration track. Referencing the mock-ups and motion instructions in the storyboard, the
animator created a draft animation of the two videos and uploaded them to Frame.io so the
design team and the clients could review them (see Figure 8).
Figure 8: View of client feedback in Frame.io--names blurred for privacy.
We continued our weekly meetings with the clients/SMEs during which we discussed their
feedback on the draft videos. I created additional storyboards of a few scenes they wanted
heavily revised. The instructional design team and I met with the motion artist to discuss these
scenes and shared the updated storyboards with him.
Once the motion artist completed full iterations of the videos that contained the narration and
sound effects, the APS team met with the clients to review them together. The clients then
shared the drafts with their larger steering committee, and directly solicited feedback from
them.
19
Product Implementation
Once the module was complete, my supervisor uploaded it to CDIAC’s education portal which
APS manages, and turned it on. To help raise awareness of the new module, CDIAC designed a
banner ad and APS placed it on the portal so existing learners would see it (See Figure 9).
CDIAC also asked APS to email an announcement about the new module to already-registered
learners. Lastly, CDIAC, through the office of the California State Treasurer, issued a press
release (see Appendix A).
Figure 9: The banner ad announcing the new module in the CDIAC portal.
Assessment of Student Learning
Gathering user data for the module was outside the bounds of this project for APS. Working
around this constraint, I conducted UX testing with a test user. This user fit into the
demographic range of learners: 47, male, college-educated, but without much experience with
municipal finance. One way the user differed from the demographic is that this was his first
experience with any of the CDIAC modules, whereas most of the other learners will have
completed at least one other module.
The test user got three of the eight items incorrect. The passing score is set at 80%, which
means that users must get seven questions correct to pass. However, CDIAC wants to
encourage users to pass the module. The module’s settings allow users that do not pass on
their first attempt to navigate back to the items they got incorrect and reattempt the items until
they achieve a passing score.
20
The test user experienced some confusion with how the multiple response items were scored.
He noted that even after viewing the item feedback he was still unsure of which answers were
correct: it was not clear to him that the responses outlined in green should have been selected
while none of the responses outlined in red should have been selected (see Figure 11).
Figure 10: Assessment Item Feedback
The development process of these CDIAC modules does not include a formal step for assessing
learner outcomes: all of the formal assessment and evaluation of the product occurred before
the product was released. As a junior employee joining this multi-year project, I did not feel it
was my place to suggest significant changes to the development process that would add time
and cost. The clients created the assessment questions themselves and while APS offered
feedback it was understood that the questions were to remain mostly the same.
Evaluation
This project was particularly very client-driven; because of this CDIAC, rather than the
designers, conducted the product evaluation. Our clients had their larger steering committee
review the module when it was semi-complete: the videos had been combined with the
assessment items, but the final sound effects had not yet been added. Our CDIAC clients sent
a review link to additional CDIAC staff (both finance professionals and elected officials) and a
committee of 12-20 external stakeholders from across California. This committee has previously
approved the scripts that the videos were created from so they are familiar with the
content. Based on their feedback, we reworked the animation for the CAB section: switching
the imagery from a helium balloon to a water balloon to increase the sense of foreboding.
Having made that update, the module was released. There are no plans to conduct any further
formal evaluations.
With the summative product evaluation review handled by the client, I focused on an internal
evaluation. I created a rubric for evaluating these aspects of the module: cognitive load,
21
learner engagement, as well as my work as a designer. My supervisor, Kerrie Dunn, has been
involved with the CDIAC project since the beginning so she has both an intimate knowledge of
the project’s history along with my contributions. The rubric and her responses are shown in
Appendix B.
Annotated Bibliography
Domain Knowledge
Before I began work on this project I was largely unfamiliar with the size and scope of the
economy of California. Researching California’s economy helped me better understand the
learner needs and the stakes at play for elected officials to make financially prudent municipal
bond decisions.
Economy of California. (2023, February 3). In Wikipedia.
https://en.wikipedia.org/wiki/Economy_of_California#:~:text=The%20economy%20of
%20the%20State,Germany%20and%20ahead%20of%20India.).
This article helped me understand the scale of California’s economy, especially in relation to
other economies around the world.
California State Treasurer. (n.d.). About CDIAC. CDIAC Introduction. Retrieved February 3,
2023, from https://www.treasurer.ca.gov/cdiac/introduction.asp
This website helped me understand what CDIAC was and learn about their organizational
mission.
I was especially interested in what the research recommended when it came to designing
instructional videos. I knew that because my project was one module of a larger in-progress
project I would be limited in what changes I would be able to make, but it was important to me
to understand what best practices were generally agreed to be.
Brame, C. (2017, October 13).
Effective Educational Videos: Principles and Guidelines for
Maximizing Student Learning from Video Content
. CBE—Life Sciences
Education 2016 15:4. Retrieved April 18, 2023 from
https://www.lifescied.org/doi/full/10.1187/cbe.16-03-0125
This article was helpful to me as I sought to translate cognitive load theory into specific
elements of instructional videos. The article suggests that designing mindfully with the
elements of 1) cognitive load 2) student engagement 3) active learning in mind can help
instructors maximize the benefits of instructional material. For the rubric I created I used
metrics that fall into these three categories. I especially appreciated the list of specific choices
Brame listed in each of these three areas. The suggestion that multiple videos in a lesson
should be six minutes or less had a major impact on my proposal to revise the module format
and chunk the video into eight smaller segments rather than two 15-minute segments.
22
Murdock, Kelly L., "Exploring Heuristics and Best Practices for Redesigning Instructional Video"
(2022). Theses and Dissertations. 9749. Retrieved February 2, 2023 from
https://scholarsarchive.byu.edu/etd/9749
I was pleased to find a dissertation on instructional videos by Kelly Murdock, a Ph.D graduate of
the IP&T program. Murdock’s discussion of different heuristics of instructional video and the
research that had been done into them, notably by Guo et al, was very helpful to me. This
article was the first one where I came across Guo et al’s recommendation of videos being six
minutes or shorter which had a major impact on my design decisions.
Learning Theory
Studying Cognitive Load helped me empathize with my learners and understand the processes
they would be going through.
Clark, R., Nguyen, F., Swell, J. (2005). Efficiency in learning: evidence-based guidelines to
manage cognitive load. Pfeiffer. Retrieved from https://learning-oreilly-
com.byu.idm.oclc.org/library/view/efficiency-in-learning/9780787977283/
This book helped me understand how learners construct schemas to incorporate information
into their memory. It also emphasized how experts in a topic will have built up capacity for
larger schema but beginners need to construct simpler schema. This influenced my decision to
advocate to chunk up the videos into small segments and also explained some of the initial
push-back we received from our SMEs. Because the SMEs already have complex schema in
place for the content they provided for the module, it seemed much simpler to them than it
would to a novice. This book also made me aware of the different types of cognitive load my
learners would be experiencing: intrinsic, germane, and extraneous and how important it was to
balance the three types.
Instructional Design Approaches
Merrill, M. D. (2018). Using the First Principles of Instruction to Make Instruction Effective,
Efficient, and Engaging. In R. E. West, Foundations of Learning and Instructional Design
Technology: The Past, Present, and Future of Learning and Instructional Design
Technology. EdTech Books. Retrieved from
https://edtechbooks.org/lidtfoundations/using_the_first_principles_of_instruction
Merrill’s First Principles of Instruction were helpful to me as I brainstormed how I would like to
design the module. I agree with the idea that including problem-based learning is beneficial to
learners as he advocates, and I would have liked to include some scenario-based assessment
items in the module. However, business constraints meant that I unable to deviate from the
existing format in that way.
Hill, J. & Jordan, L. (2021). Instructional Strategies. In J. K. McDonald & R. E. West (Eds.),
Design for Learning: Principles, Processes, and Praxis. EdTech Books.
https://edtechbooks.org/id/instructional_strate
Hill and Jordan emphasize the important and benefits of including relevant material in
instruction. Because this module is for all elected officials in California: school district officials,
23
county officials, city officials, state officials, etc.…the module by necessity omits discussing
specific scenarios. However, if we prioritized the principles in Hill and Jordan’s work, structuring
the module to include specific scenarios for each of these official types, perhaps in a branching
format, would be something to consider.
Design Knowledge
It sounds trite, but for me the most important takeaway from this project is the importance of
communication. I learned that it did not matter how much research I had done to back up my
proposal to divide the module videos into small chunks if I did not communicate well with the
client. However, having done the research was critical to advocating for my position. When I
first mentioned the idea of chunking the videos to the clients, their knee-jerk reaction was to be
against the idea. Once my supervisor and I took the time to explain that according to research,
videos were less effective if they were longer than about six minutes, they began to be more
open to the idea, but they were still not enthusiastic about it. I felt strongly about the idea and
so it was worth it to bring it up again at the next meeting we had. There were some awkward
pauses as they considered the idea, and I felt an urge to jump in and tell them “never mind!”
But I resisted and waited for them to speak because I felt strongly that this change would
improve the product for the learners. The awkwardness was worth it because they agreed to
try out this new format which I believe will benefit the learners.
Communication was also key in managing the exchange of feedback. It was important to
accurately capture the clients’ feedback and then package it in an easy-to-understand format
before sending it on to the animator. It was not enough just to record the feedback; I had to
put effort into ensuring it was understood correctly to avoid delays and additional cost.
Working well with feedback was also important when it came to creating quick iterations to
keep the project momentum going, something I learned from the IP&T project management
class and tried to apply here.
Design Critique
It was at times frustrating to not be able to apply other things I had learned during my IP&T
coursework to this project: namely, not being able to include problem-based learning or
conduct much formal evaluation. I often felt pulled between “doing it right” and “getting it
done.” However, as I plan to continue working as an instructional designer in the corporate
sector I think I’m likely to feel this tension often, so this was good practice. While I was not
able to make all of the design choices I wanted to, the experiences I had learning to balance
best practices with real life business situations was valuable in and of itself.
Conclusion
I thoroughly enjoyed my time in the IP&T program, and the only reason I was able to
successfully complete this project was by relying on the skills and knowledge I gained there.
One of key takeaways I took from the program is the importance of doing the pre-planning
work: learner and environmental analyses, research, prototypes, etc.… Being prepared helped
calm the nerves I felt going into a new situation. Being prepared helps me feel confident that I
24
can handle the hiccups that will inevitably occur in the real business world. Having done this
pre-planning work also helped me justify or translate my design decisions for clients without
getting (very) flustered.
I have enjoyed school since I was a child and returning to BYU as a non-traditional student to
earn a master’s degree was the achievement of a dream. Besides my brother, I am the first in
my family to earn an advanced degree. My father was the first in our family to earn an
undergraduate degree, which he did with support from the G.I. Bill. My mother is an immigrant
and did not have many opportunities to pursue much formal education, but she yearned to earn
her G.E.D. and continue on in school.
Witnessing my mother’s desire to improve her education and her frustration at the obstacles
that stood in her way are a formative part of my identity as a designer. Being her daughter has
heightened my awareness of the importance of learning opportunities for all people. I am
grateful to the IP&T program for the skills and experiences it has empowered me with so I will
be able to design learning solutions to help people from all walks of life enrich their lives.
25
Appendix A
Press release announcing the release of the module
California State Treasurer Fiona Ma, CPA
News Release
FOR IMMEDIATE RELEASE
PR 23:20 Contact: Joe DeAnda
May 10, 2023 (916) 653-2995
State Treasurer Fiona Ma Releases Latest Municipal
Finance Training Module for Elected Officials
SACRAMENTO – State Treasurer Fiona Ma today released the latest municipal
finance training module designed for elected officials. The module, titled Marketing
and Structuring Strategies for a Win-Win Debt Issuance, is the sixth in the Elect>Ed
Debt Issuance and Administration series designed to help elected officials navigate
the complex and critically important world of municipal finance and debt issuance.
“Effectively using and managing debt is critical for local governments,” said
Treasurer Ma. “This new module continues my office’s efforts to ensure that
California decision makers at all levels have the resources they need to make
informed decisions on behalf of all Californians.”
26
Elect>Ed modules are produced by the California Debt and Investment Advisory
Commission (CDIAC), which is chaired by Ma as state treasurer. The courses are
offered through CDIAC’s Ed Portal, a learning management system that delivers a
self-directed learning experience and creates the means to track the program’s
effectiveness in meeting the learning objectives of the target audience.
Previous modules in the series include:
Module 1 | Why Do Public Agencies Use Debt Financing (15 mins)
Module 2 | What You Need to Know Before Issuing Debt (13 mins)
Module 3 | Strategy Drives Structure (25 mins)
Module 4 | The Use of Debt Policies to Manage Risk (22 mins)
Module 5 | It Takes a Team: The Role of External Professionals in a Successful
Issuance (20 mins)
Module 6 | Marketing and Structuring Strategies for a Win-Win Debt Issuance (29
mins)
A seventh module, The Commitments You Make: Implications and Documentation
of Long- Term Debt, is expected later this fall.
CDIAC has also released Part 1 and 2 of the Elect>Ed Pension Fundamentals
Series:
Part 1 | Pension Introduction (27 mins)
Part 2 | Elected Officials’ Role in Pension Management (40 mins)
All modules are free for decisionmakers and can be accessed online:
https://edportal.treasurer.ca.gov
There are approximately 4,400 local government agencies in California, which
include cities, counties, K-14 districts, and special districts and are represented by
thousands of local officials. About 60%-70% of local governments issue some kind
27
of debt. CDIAC improves the practice of public finance in California by providing
responsive and reliable information, education, and guidance to state and local
public agencies and other public finance professionals.
###
Fiona Ma is California’s 34th State Treasurer. She was first elected on November 6, 2018,
with more votes (7,825,587) than any other candidate for treasurer in the state's history and
reelected on November 8, 2022. She is the first woman of color and the first woman
Certified Public Accountant (CPA) elected to the position. The State Treasurer’s Office was
created in the California Constitution in 1849. It provides financing for schools, roads,
housing, recycling and waste management, hospitals, public facilities, and other crucial
infrastructure projects that better the lives of residents. California is the world’s fifth-
largest economy and Treasurer Ma is the state’s primary banker. Presently, her office
processes more than $3 trillion in banking transactions. She provides transparency and
oversight for the government’s investment portfolio and accounts, as well as for the state’s
surplus funds. Treasurer Ma oversees an investment portfolio that currently averages over
$200 billion—a significant portion of which is beneficially owned by more than 2,200 local
governments in California. She serves as agent of sale for all State bonds and is trustee of
billions of dollars of state indebtedness.
###
For more news about the State Treasurer’s Office, please follow Treasurer Ma on
Twitter at @CalTreasurer, Instagram at @CalTreasurer, and on Facebook at
California State Treasurer's Office.
28
Appendix B
Evaluation Response by Kerrie Dunn, Director of Instructional Design at APS
Metric
1-5
1= Low
5=High
Comments If Any
Cognitive load
Learner Engagement
Active Learning
Professionalism
