Exploring Literate Programming in Electrical Engineering Courses

Francis Igot — Thu, 03 Dec 2020 13:00:00 +0000

Bryan A Jones ¹ , J W Bruce ² , Mahnas Jean Mohammadi-Aragh ¹

1 Department of Electrical and Computer Engineering, Mississippi State University, Mississippi State, MS, USA

2 Department of Electrical and Computer Engineering, Tennessee Technological University Cookeville, TN, USA

Abstract

Knuth’s literate programming paradigm positions source code as a work of literature for which communication to a human is prioritized over communication to a computer. A primary pedagogical value of literate programming lies with the act of writing, especially good writing, leading to good thinking. Issues with early literate programming tool implementations plagued the classroom adoption of literate programming. Advances in technology have warranted a reinvestigation of the benefits of the paradigm. To complement existing inquiry of literate programming in computer programming courses, we investigate, “How can literate programming support student learning in microprocessors and digital system design courses?” In our examination of microprocessors, the instructor used principles of literate programming during in-class demonstrations of assembly programming. In our examination of digital system design, students used the tool to engage in literate programming while writing in a hardware description language. Our results indicated students had a slight preference for instructors to utilize literate programming when presenting in-class examples, and we observed small improvements for graded assignments in sections in which literate programming examples were employed. We also observed a difference in preferences for literate programming by major (computer versus electrical engineering) and noted multiple instructor-observed challenges with introducing a drastically different pedagogical technique in upper-level courses. While our examination did not produce statistically significant results, student and instructor perceptions can be used to guide future literate programming implementations and investigations.

Keywords

Literate Programming, Computing Education, Writing

Related ASEE Publications

1, 2, 3

Introduction

A glance at current events shows our society’s dependence on software. In late January 2016, Nest, Inc., which developed and produces a thermostat controllable via smartphones and accessible via WiFi, pushed a software update to all their devices. Unfortunately, bugs in this update caused users’ thermostats to drain the battery, disabling the thermostat and leaving heaters turned off in the middle of winter 4. Angry complaints of babies waking up at 4 AM to a room at 62°F and panicked calls to restore their homes to operation flooded Twitter and on-line forums. As another example, Volkswagen now faces billion-dollar fines for “defeat device” software in its diesel cars which caused them to pass EPA emissions tests in the lab while emitting much higher pollutants during actual driving conditions5. Digital devices and the programs that control them are ubiquitous in modern life.

With advances in technology resulting in complex digital circuitry embedded everywhere in modern life, the importance of programming and digital design knowledge continues to grow. Computer science, computer engineering, and electrical engineering students are expected to master the art of programming and learn multiple domain-specific languages including Python, C++, MATLAB, and R to perform analysis across a number of engineering disciplines. Further, digital system designers often need to use hardware description languages (HDLs) such as ABEL 5, VHDL (now described by IEEE Standard 1076), and Verilog (now described by IEEE Standard 1364). These HDLs are used to describe both behavior and structure of a digital circuit, serving as documentation tools that possess characteristics similar to traditional computer programming languages. The introduction and use of modern HDLs, predominantly Verilog and VHDL, are a hallmark of modern computer engineering curricula 6. However, most engineering students have no prior exposure to such languages and, as least initially, find HDLs challenging. Students can be further confused when learning Verilog, as Verilog is syntactically similar to the sequential procedural C programming commonly taught to engineering students.

In spite of the pressing need for capable, creative, and above all competent programmers and digital device designers, educators struggle to effectively train students in these essential skills. McCraken’s comprehensive examination of first-year CS students reports that only approximately 20% of the surveyed students could solve programming problems expected by their instructors 7. “Issues impacting students learning how to program” was the topic of an entire Computers in Education Division technical session at the 2019 American Society for Engineering Education (ASEE) Annual Meeting 8. Clearly, there is a need to explore new pedagogical approaches for teaching students how to program and design digital systems.

Authors Jones and Mohammadi-Aragh are actively exploring literate programming (LP) as one approach to improving programming pedagogy 9, 2, 1. In the LP approach, the programmer (author) composes the program (document) in a form that is readable by humans. In short, the program should be like literature, an essay that contains both explanation for human readers and executable statements for compilers 10. Jones and Mohammadi-Aragh’s exploration of LP in introductory programming courses produced positive results 3. Due to similarities between programming languages and HDLs, we were motivated to consider whether literate programming could improve student experiences learning programming and HDLs in hardware-focused courses. This work was driven by two initial questions: 1) How could LP principles be implemented in hardware-focused course? and 2) How would students respond to LP in hardware-focused courses?

In this paper, we discuss results of our efforts to explore the use of literate programming (LP) methods in two electrical engineering courses: a microprocessors and a digital system design course. Our implementation of microprocessors required students to program in C++ and PIC24 Assembly. Our implementation of digital system design required VHDL.

Background Literature

Literate Programming (LP)

Donald Knuth proposed LP 11 in 1984. In the literate programming paradigm, code and comments are equally important. The literate program mandate that a programmer’s creative output consist of a document which contains intermingled code and prose differs radically from typical software development models that view the creative output as code in which scattered comments are not required to connect into a coherent whole. A quick glance at the current state of computer programming paradigms will reveal that Knuth’s efforts in LP were not widely adopted. It has been suggested that the failure of adoption of LP for pedagogical use is due, in part, to the lack of sophistication of early LP tools10 . This last point is substantiated by noting that most education-focused research using LP tools took place in the 1990s. Efforts in this area include using LP tools to grade homework submissions 12 , teach programming13 , or write better comments 14 . The work in 14 and 15 reports some success, but both cite student complaints about the difficulty of using LP tools.

Knuth’s initial LP implementation, called WEB, incorporated ideas of LP but had major weaknesses that limited adoption. The first weakness was related to language support. Pascal was the only supported language. The second weakness was related to writing and formatting. WEB used macros and preprocessor directives to create a meta-language “source document” that would render a WEB file into two separate outputs: a human-readable document and source code for compilation. The WEB source document consisted of a series of cryptic formatting instructions (Figure 1(a) ), some of which allowed the inclusion of Pascal code. Knuth’s choice of an additional source document meant that the formatted human- readable document (Figure 1(b) ) and the source code (Figure 1(c) ) had to be created by WEB. Neither the rendered text nor the source code could be edited directly and minor textual edits become major chores. Making minor edits to the formatted document in Figure 1(b) required finding the text in the source document in Figure 1(a) which produced it, editing the offending lines, then regenerate the formatted document to verify that the correct edit was performed. Likewise, modifying the source code in Figure 1(c) requires a similarly laborious process. The “source document” design choice completely isolated authors from their writing. Finally, a third weakness with WEB was that traditional development tools such as debuggers and profilers were extremely difficult to deploy for WEB documents and their associated programs.

Figure 1: Knuth’sWEB system for LP transforms the input source document in (a) to the formattedoutput in (b) and the source code in (c) as illustrated by the large arrows

Today’s popular document generators do not easily allow elaboration about the internal workings of a program — the primary goal of LP. LP implementations developed after WEB somewhat addressed WEB’s weaknesses in language support and formatting. Some variants support additional programming languages: CWEB (for C), FWEB (Fortran, C, and C++), xmLP (XML), pyWeb, FunnelWeb, nuweb, and noweb (any language). Other tools provided simpler formatting syntax: nuweb uses LaTeX; noweb uses a simpler set of LP directives and also allows use of LaTeX; pyWeb uses restructured text. Pieterse’s survey reviews these and other LP approaches 12. More recently, documentation generated directly from code has become widespread. Documentation generators, such as Doxygen and JavaDoc, produce a document directly from formatted comments within source code, thus overcoming WEB’s second problem. These document generation tools are widely used by programmers, with thousands of programs employing these tools or variants. However, while tools like Doxygen, JavaDoc, and others, were inspired by LP principles 16, they are currently only used to document a program’s external interface (e.g., application programming interface (API)). Investigations into the use of documentation generators to support LP principles are needed.

Cognitive Load Theory

Knuth’s LP paradigm is consistent with cognitive load theory 17, which states that keeping related concepts close, temporally or spatially, can improve the ability of students to grasp difficult ideas. Sweller’s Cognitive Load Theory17, 18, 19 posits that new concepts (termed schema) must be learned in the limited capacity of short-term memory before they can be stored in and employed by long-term memory. The cognitive load of acquiring a new schema can be categorized into several distinct loads. When a schema consists of multiple, interconnected ideas, they must all fit in the limited capacity of short-term memory for learning to occur. The intrinsic cognitive load refers to the number of interconnected ideas which are required to learn a new schema. Poor instructional design can impose extraneous cognitive load, in which the instructional technique unnecessarily increases the number of interconnected ideas, making a schema harder to acquire. When intrinsic load is low, the additional extraneous loading has little effect on learning. However, high intrinsic loading combined with extraneous loading hinders learning. Therefore, instructors should strive to reduce extraneous load produced by their presentation of the material to improve student learning. LP supports instructor efforts by positioning student writing consisting of design decisions, formulas, and figures in close proximity to the source code they are writing.

CodeChat: A Modern LP Approach

One of the authors of this paper has developed a modern tool — CodeChat 20 — that attempts to address WEB’s failings while still incorporating the main premise of LP. The CodeChat tool combines the strengths of both documentation generators and LP tools. CodeChat builds a formatted document directly from source code, using human-readable ReStructuredText 21 as markup contained in the language’s native comments. CodeChat contains a synchronization mechanism which matches source code with the corresponding web output, making editing of either straightforward. CodeChat users simply edit a programming language source file in the left pane, shown in Figure 2(a) and Figure 3(a). The user places very intuitive ReStructuredText markup in the language’s comment lines. The CodeChat tool parses the source code file, locates the markup, and renders the program and annotations into human-readable form in real-time into the right-hand pane (see Figure 2(b) and Figure 3(b)). The rendered output can be generated in many different formats, including HTML, TeX, LaTeX, and DocBook. This single-view paradigm is understood by today’s computer users within a few moments of use. CodeChat provides an easy- to-use and viable platform for LP techniques in programming. CodeChat supports LP in more than 200 programming and scripting languages “right out of the box.” Additionally, it is OpenSource and freely available, making CodeChat a viable tool for conducting research into the use of LP in programming education pedagogy. Figure 2 and Figure 3 illustrate the application of CodeChat in the two courses examined for this study.

Figure 2: CodeChat, the literate programmingimplementation used to conduct research for this paper, transforms traditionalmicroprocessors course source code in (a) into the web page shown in (b) as shown by the arrow.

Figure 3: CodeChat transforms digital system design HDL in (a) into the webpage shown in (b) as shown by the arrow.

Research Overview

We assert that good writing leads to good thinking, and good thinking to good programs. We believe that advances in technology and user interface design have warranted new explorations of the benefits of Knuth’s LP. We revive Knuth’s ideas by developing a new LP tool addressing weaknesses for prior implementations. Then, we use our tool in two electrical engineering courses to answer: “How can LP support student learning in microprocessors and digital system design courses?” In our examination of LP in microprocessors, the instructor used the tool to demonstrate LP during in-class exercises. In our examination of LP in digital system design, the students used the tool to engage in LP for homework and in a design lab environment.

We investigated the impact of LP in a microprocessors course because the course requires students to integrate material from multiple sources creating high cognitive load. For example, the function shown in Figure 2(a) also depends on understanding of a timer and specific reference to the definition of bits in timer 2’s control register which is provided in the microcontroller’s datasheet. Cognitive load theory predicts that when these elements are spatially or temporally separated, such as refer- ring to a textbook, a datasheet, and traditional source code, additional extraneous load is imposed to successfully integrate these elements. Because LP encourages including all these elements as a part of the document, as shown in Figure 2(b), we hypothesize that the use of literate programs will reduce extraneous load, thereby improving students’ ability to master these concepts, which will lead to higher test scores.

We investigated the impact of LP in a digital system design course because modern hardware description languages have similarities with programming languages, and there is limited research investigating how LP can support students using hardware description language. The introduction and use of modern HDLs, predominantly Verilog and VHDL, are a hallmark of modern computer engineering curricula [3]. Because digital devices operate concurrently, HDLs allow for description of concurrent operations, similar to programming languages like Haskell and Ada. The complexity of hardware descriptions coupled with HDL similarities to sequential programming languages has led us to propose the idea of introducing LP into HDL education. Using LP in HDL education also reinforces the spirit of why HDLs were created in the first place – to describe hardware behavior. An HDL description should describe – in the most human-readable terms as possible – what the hardware design “looks like” or “how it behaves”. The HDL description is, first and foremost, an essay written to fellow designers describing digital hardware. This description also happens to be in an “executable form” thanks to the HDL tool chain. However, the use of LP to improve HDL education has not been widely investigated. One early attempt involved an approach very similar to Knuth’s WEB approach 22, 23 using a Prolog logic program to generate a human-readable form and a Verilog HDL file. Our investigation provides further insight into the benefits of LP in support of HDLs.

CASE 1: Microprocessors Course

The microprocessors course used for this study focused on introducing students to micro-processors through both lecture and laboratory exercises. The first half of the course focused on instruction in assembly language, and was accompanied by labs in which students translated a C program to assembly, then simulated their assembly code to demonstrate its correctness. The second half of the course required students to build a simple microcontroller and supporting components on a wireless protoboard, then to develop C programs to interface with external peripherals connected to the microcontroller. The first two examinations assessed assembly skills, while the third test and portions of the cumulative final assessed peripheral interfacing in C. Students typically perform well in introductory assembly instruction assessed on Test 1, then struggle with more advanced assembly, such as pointers and extended-precision operations, assessed on Test 2. Likewise, introductory peripheral interfacing in C assessed on Test 3 typically produces good scores, while students struggle with more advanced concepts (I2C and SPI buses, A/D and D/A converters) and their implementation in C on the final.

We hypothesized that LP would improve comprehension as the semester progressed to more challenging content distributed across multiple resources. Test 1 material, consisting of introductory assembly, can typically be taught by focusing exclusively on the code itself. While LP provides better formatting and organization, little gain should be expected from a cognitive load perspective because the source code itself contains all the necessary resources. Test 2 material, particularly when discussing pointers, exacts a heavier cognitive load. Students must refer to a memory map (in the form of a table), carefully examine the C code to translate, then write the resulting assembly code. Therefore, the ability of LP to integrate memory maps into the source code should reduce extraneous cognitive load and improve comprehension. While some Test 3 material relies on information available within the source code, some does not. For example, to fully understand the operation of code to configure a timer in Figure 2(a), students must refer to the microcontroller’s timer documentation. This higher cognitive load should produce reduced comprehension using traditional techniques, since students’ attention must be split between the code and supporting datasheets. Final material includes relatively complex bus protocols such as I2C and SPI, both of which must be used to interact with peripherals. The ability to integrate related information with the code should reduce extraneous cognitive load, producing improved comprehension and scores. For example, comment 162, “Send address of temperature LSB (0x01)” in Figure 2(b) is immediately followed by the table of register addresses, which gives the temperature LSB address as 0x01. Therefore, the following line of code, ioMaster- SPI1(0x01);, clearly sends the temperature LSB address of 0x01 across the I2C bus.

Approach for Microprocessors

Our investigation made use of a cohort of 58 students enrolled in two sections of the microprocessors course during the Fall 2015 semester. Students self-selected their preferred section based on course scheduling (i.e., timing) preferences. One section, the control group of 27 students, employed no LP methods. The other section of 31 students alternated use of traditional with LP techniques. Specifically, instruction using traditional coding methods was employed for Test 1 material, by developing and discussing code snippets in assembly during in-class exercises. All code written during class was then uploaded to Github 24, a social coding website, providing convenient web access to the code for the students. Next, material for Test 2 was developed using LP techniques to produce a document during in-class exercises. Both the resulting assembly code and its rendering to a LP document were posted on Github and the course website 25, respectively. The same approach was taken for the second half of the course, which employed the C programming language. Test 3 material was covered using traditional techniques, while the material for the final was developed as a set of LP documents. For both cases, the C source and the resulting LP documents were available via the web. Figure 4 shows a comparison between traditional source code (in this case, assembly) used for Test 1 in-class exercises and its LP form used for Test 2 instruction. Likewise, Figure 2 compares traditional C code for Test 3 preparation with the LP variant for the final.

Figure 4: Traditional assembly code in (a),compared to its literate programming form in (b).

Data Collection for Microprocessors Case

To evaluate student performance, specific questions were selected on the tests and final which require writing snippets of code in C or assembly. In addition, students in the experimental section participated in an anonymous in-class survey given at the end of the course to provide feedback on the use of LP versus traditional code in the course. The questions were:

How often did you refer to code discussed in class posted at https://github.com/bjones1/ece3724_inclass? Frequently, several times, a few times, once, or never?
On a scale of 1 to 10, where 1 is the least helpful and 10 is the most helpful, how helpful was providing this code on the web?
Comments – what would make the code posted more helpful?
How often did you refer to the code discussed in class posted in LP form at http://courses.ece.msstate.edu/ece3724/in_class? Frequently, several times, a few times, once, or never?
On a scale of 1 to 10, where 1 is the least helpful and 10 is the most helpful, how helpful was providing this code on the web?
Comments – what would make the code posted in this form more helpful?
Which format did you find more helpful? Traditional, literate programming, neither, or don’t care
In terms of the code discussed, what would help you learn more during this course?

Results for Microprocessors Case

Table 1 shows the test questions used to examine student performance. Note that the control section only employs traditional source code, while the experimental section alternates between traditional and LP. Through the differences reported between the control and experimental sections, we attempt to control for the effects of confounding variables between the sections such as instructor style, student background, and student ability. The mean differences (∆_M ) show a statistically insignificant value of 3% between the two sections for Test 1, during which both sections employed traditional source code. This suggests that effects of confounding variables are small, particularly given the wide standard deviations. For Test 2, which compares traditional to LP instruction, a slightly larger difference of 8% was observed. Treating Test 1 as a baseline difference and Test 2 as the expected effect shows a vertical difference of 5%, suggesting a small, but statistically insignificant improvement using LP. The Test 3/Final comparison suggests confounding variables have a greater impact, with a 14% difference for traditional source code. Further, the effect of LP was a statistically insignificant decrease of 1%.

Table 1: Microprocessors Test Scores

Assessment	Control	Experimental		∆M
	Traditional (M ± SD)	Traditional (M ± SD)	LP (M ± SD)
Test 1, #12-20	91% ± 8	94% ± 12		3%
Test 2, #17-28	69% ± 13		78% ± 17	8%
Test 3, #1-11, 16-29	70% ± 15	84% ± 14		14%
Final, #24-29	58% ± 19		72% ± 22	13%

Students could provide multiple free-form answers to the eight survey questions listed in the previous section. Thus, the survey results were coded based on the categories provided in the questions. Table 2, Table 3, Table 4 show a summary of the frequencies reported for questions 1, 2, 4, 5, and 7. Based on these survey results, students refer to both traditional source code and its LP form with essentially equal frequency. The higher response of 61% for traditional versus 42% for LP in the category of “a few times” represents a difference for students who made little use of either system; the usage frequencies for “frequently” and “several times” are almost identical. Likewise, noting the wide standard deviation, students considered both traditional and LP forms equally helpful. Students reported preferring the LP format over the traditional format; however, given a large group (27%) of “don’t care” responses, this is a mild preference. Given that this is the first exposure students have to the LP format, and noting that all their previous instruction and programming assignments employ the traditional form, students seem willing to embrace the LP approach. This preference is also notable in that several students reported they were unaware of the existence of the LP webpages.

Table 2: Microprocessors Question 1 and 4 Results

How often did you refer to …?	Q1: Traditional (n=38)	Q4: LP (n=24)
Frequently	3%	4%
Several Times	16%	17%
A Few Times	61%	42%
Once	11%	13%
Never	11%	25%

Table 3: Microprocessors Question 2 and 5 Results

How helpful?	Q2: Traditional (n=26)	Q5: LP
m ± S.D.	7.6 ± 1.5	7.0 ± 2.1

Table 4: Microprocessors Question 7 Result

	Tradition	LP	Neither	Don’t Care
Q7: Preferred which format? (n=22)	27%	41%	5%	27%

Analyzing the comments provided in response to questions 3, 6, and 8 produced the following four themes:

Students liked the code organized by topic, rather than class date. This suggestion has been implemented for the Spring 2016 semester.
Several students stated that they were not aware that the literate programming pages were available on the web, thinking they were only provided for in-class exercises. This response was provided even though students were sent several e-mails giving the address of the LP pages. For the Spring 2016 semester, a change was implemented that required students to access the LP website for every in-class assignment.
Students requested live updates as the instructors add examples and notes to the LP document in class. Previously, the LP document wasn’t posted to the web until after each class period. This suggestion was implemented for future course offerings.
Students requested more explanation and more videos. The class is “flipped,” which causes some students to feel cheated that lectures focus on in-class exercises, rather than delivering facts. In addition, some of the older videos need to be updated.

Discussion of Microprocessors Case

The results of our examination of LP in a microprocessors course indicate that literate programming examples may provide some benefit to students. In particular, a student survey shows a mild preference for the LP form when compared to the traditional form. In addition, an analysis of student performance on examinations shows a small, statistically insignificant improvement in student performance when the instructor employs LP for in-class examples. Larger datasets are needed to verify that this effect holds more generally. We note that in this implementation, students were not writing their own code using LP principles; instead, LP was used during the presentation of in-class exercises. A future examination of learning outcomes when students are writing their own literate programs in microprocessors is warranted and may result in expected learning gains.

CASE 2: Digital System Design Course

The digital system design course used for this study was a split-level course targeted at senior undergraduate and introductory graduate students. The course reviewed and revisited digital logic design topics from a prerequisite introductory course and added complexity and practical aspects not covered in the earlier course. All of the coursework was captured in VHDL. The course concluded with a small design project. The first half of the course focused on instruction in VHDL syntax and was accompanied by laboratory sessions in which students wrote descriptions — mostly structural — of simple hardware designs in VHDL. Students composed appropriate test benches to exercise and validate their designs. The second half of the course focused on timing and system design issues along with a more behavioral approach to hardware description. Weekly lab assignments required the students to design ever-larger components that could be reused in their final design project. The course offering described herein was similar to previous semesters with nearly identical topical coverage and pacing using the same text, lecture materials, and lab facilities. The previous iterations of the course were taught by another faculty member who did not employ LP methods. The course is offered once per academic year in a single section.

Our goal with incorporating LP in a digital system design course is to promote HDL “source code” as, first and foremost, an essay written to fellow designers describing digital hardware, which also happens to be in an “executable form” thanks to the HDL tool chain. Furthermore, the more expressive nature of human language and the student’s experience with human language should allow the students to more clearly express the behavior and interactions of the digital hardware. Using LP in HDL education reinforces the spirit of why HDLs were created in the first place – to describe hardware behavior. An HDL description should describe in as human-readable terms as possible what the hardware design “looks like” or “how it behaves”. The goal was to promote the view that hardware description language “source code” is a product for human consumption instead of tool-chain consumption.

Approach for Digital System Design

Our investigation made use of a cohort of 36 students enrolled one section of the course during a recent fall semester. The cohort was composed of ten electrical engineering majors who took the class as an elective and twenty-six computer engineering majors for whom the course is required. The course was composed of lecture periods (twice a week; one hour each) and a weekly lab session (once a week; two hours per week). The staffing included a graduate teaching assistant to assist students. Lecture periods were sprinkled with collaborative active exercises. Students were encouraged to work in teams on the lecture active exercises and on laboratory tasks. The instructor often had to expressly direct the collaborative effort as most students would choose to work alone if given the choice. The complexity of VHDL and the overhead of tool chain processes largely prohibited the use of the synthesis tools in-class for the active exercises.

In-class active exercises were mostly focused on the design approach for a particular problem, with students writing “pseudo-code” VHDL. Homework and lab activities consisted of students writing VHDL descriptions annotated with explanations in the CodeChat tool. In previous semesters, students wrote formal lab reports to describe their design approach and results. The addition of LP annotations to their VHDL was the only significant change to VHDL coding assignments compared to previous semesters. The CodeChat tool allowed students to include hyperlinks, figures, tables, equations, FSM diagrams, timing waveforms, etc. directly in their HDL descriptions. Since the code itself was descriptive, and each design’s test bench could be annotated with results obtained from the VHDL tools, students were not required to submit a formal lab report. Each design task had a deliverable of VHDL files that were to be liberally annotated with descriptions, explanations, figures, timing diagrams, and observations.

Data Collection for Digital System Design Case

To ascertain whether LP had an impact on the students’ ability to successfully learn VHDL and capture digital systems behavior, at the conclusion of the course students were asked to respond to a survey. Survey questions are provided in Table 5 . Students responded to the survey questions using five-point Likert-type scales. For questions 1 and 2, students could choose a response from the following list: poor, fair, satisfactory, very good, or excellent. The responses were mapped to the values zero (poor) through four (excellent). For questions 3-6, students were given the choice of strongly disagree, disagree, neutral, agree, and strongly agree. These responses were mapped to values -2 (strongly disagree) to +2 (strongly agree), which allows the sign of the response to indicate whether the response is negative or positive.

Table 5: Digital System Design Survey Questions

Question	Question Text
1	I would rate my knowledge of HDL and digital systems design knowledge at the start of the course.
2	I would rate my knowledge of HDL and digital systems design knowledge at the conclusion of the course.
3	Writing detailed and descriptive comments in my HDL descriptions helped me to learn.
4	I would have rather had traditional Q&A questions instead of writing documented HDL descriptions for course homework.
5	My digital design efforts in this class would have been better summarized with a formal report/paper.
6	Putting my design ideas into words helps me to see errors in my design and improves my overall output.

Degree program information (i.e., major) was collected to examine how differences in background affect a student’s view of LP as it was used in the digital systems design course. While computer engineering and electrical engineering are very similar, the computer science and programming skills and experience of the computer engineering majors usually is much more substantial than those of electrical engineering students.

Results for Digital System Design Case

Table 6 provides the statistics of the students’ self-assessed abilities by major for Q1 (knowledge at start of course) and Q2 (knowledge at end of the course). A small number of computer engineers (four out of 16) reported they had some knowledge of the subject before the course started, while electrical engineering majors all reported no prior knowledge of the course topics. At the end of the course, most students of both majors reported their knowledge as being “satisfactory” or better. The difference between a student’s response (Q2-Q1) would indicate the course impact on their knowledge. Computer engineering and electrical engineering majors reported a mean difference between Q1 and Q2 of 1.82 and 1.86, respectively.

Table 6: Digital System Design Question 1 and 2 Results

Knowledge of DSD and HDL	CmpE (M ± SD, Mdn)	EE (M ± SD, Mdn)	Combined Cohort (M ± SD, Mdn)
Q1: Start	0.29 ± 0.59, 0	0.00 ± 0.00, 0	0.21 ± 0.51, 1
Q2: End	2.12 ± 1.22, 2	1.86 ± 1.07, 2	2.04 ± 1.16, 2
∆M (Q2-Q1)	1.82	1.86	1.83

Knowledge of DSD and HDL

CmpE

(M ± SD, Mdn)

Combined Cohort

(M ± SD, Mdn)

Q1: Start

0.29 ± 0.59, 0

0.00 ± 0.00, 0

0.21 ± 0.51, 1

Q2: End

2.12 ± 1.22, 2

1.86 ± 1.07, 2

2.04 ± 1.16, 2

∆M (Q2-Q1)

1.82

1.86

1.83

Table 7 provides a summary of student preferences by major for Q3 (writing helped me learn), Q4 (prefer traditional HW style), Q5 (prefer traditional lab report style), and Q6 (writing helps me improve my design). The results from Q3 indicate that computer engineers were slightly more negative than neutral about the LP approach. Electrical engineers were exactly neutral as a group. Of course, individual students reported “strongly agree” and “agree”, but there were roughly equal numbers of students reporting “strongly disagree” and “disagree”. In Q4, computer engineers indicated a slight preference for homework based on LP-infused VHDL over a set of traditional homework problems. Electrical engineers indicated an even stronger preference for LP and VHDL, although neither group was overwhelming in their preference. Not surprisingly, Q5 responses indicate that both majors are more emphatic about preferring the LP approach to VHDL compared to writing a formal lab report. As with Q4, the preference among the electrical engineers was stronger than the computer engineers. Q6 was worded to try to elicit the student response about LP without ascribing it to the course and lab experience in support of Q3. Both majors were effectively neutral in their views, with electrical engineering majors being slightly positive on average, and computer engineers as a group were nearly neutral. Note that response in Q6 are a bit more positive than the very similar question Q3. Students appear to recognize that writing and natural language may help them express their designs, but the LP approach or negative feelings due to CodeChat installation drove their opinions a bit more negative when the writing involves commenting their HDL.

Table 7: Digital System Design Questions 3-6 Results

LP and Writing in HDL	CmpE (M ± SD, Mdn)	EE (M ± SD, Mdn)	Combined Cohort (M ± SD, Mdn)
Q3: LP helped	-0.41 ±1.33, 0	0.00 ±0.00, 0	-0.29 ±1.27, 0
Q4: Prefer Traditional	-0.18 ±1.19, 0	0.29 ±1.25, 0	-0.21 ±1.18, 0
Q5: Prefer Formal Report	-0.65 ±1.17, -1	-1.29 ±0.76, -1	-0.83 ±1.09, -1
Q6: Writing helps	-0.06 ±1.09, 0	0.29 ±0.95, 1	0.04 ±1.04, 0

A Mann-Whitney U test was chosen as a non-parametric test to compare the two populations 26. No significant differences between the two populations were observed on questions Q1-Q6.

Discussion of Digital System Design Case

Students were somewhat ambivalent about the effectiveness of LP to improve their HDL in the digital design course with computer engineering students somewhat more negative. One theory is the computer engineers are more comfortable with the topics covered in the course and with programming languages and documentation than the electrical engineers. The design tasks in the course were not overly complex, so computer engineers may have felt the LP paradigm was simply too much overhead and work for the reasonably simple technical HDL that followed. Electrical engineers are less comfortable in the course described here, and the LP approach may have allowed them to better express themselves. If this is a valid view, one might expect the student enthusiasm to increase as assignments and designs get more complicated. Such a result would also agree with the observation that computer programmers tend to comment more heavily in complicated code, or code that they do not initially understand 27, 28. LP would provide the developer with a more suitable mechanism for human-digestible descriptions.

Students, especially the computer engineering majors who have more experience with computing languages, may have also suffered from the mistaken impression that any computer activity time spent not creating “working code” is a waste of time. With more experience, they may realize that documentation of computer code or VHDL descriptions leads to a much higher maintainability and, ultimately, shorter overall development cycles and lower costs 27, 28.

Anecdotal observations by the instructor and other faculty are that student comprehension and performance seemed largely unchanged compared to previous semesters. The next step would be to form a controlled experiment wherein one group will use traditional editors or IDEs to develop largely undocumented HDL descriptions. A treatment group would use the tools and LP approach described here. Incorporation of formal product metrics such as defect production and development time would also give more objective measures to the quality of student output.

Finally, student performance on other course measures or later academic outcomes might indicate a difference. Alternatively, an ideal, but often difficult to deploy, approach to testing the efficacy of LP in the digital systems design course would be a formal experiment with control and treatment groups of students taught under similar educational conditions during the same academic period. This was not possible due to existing scheduling and faculty workloads. Another approach could create control and treatment groups within the same lecture and lab population. This approach is problematic as students may view different assignments with different levels of difficultly or otherwise detrimental to their grade. Control and treatment groups could be formed by student self-selection, but groups of comparable sizes, abilities, and motivation are very suspect with this scheme. Given the circumstances, a decision was made to apply the treatment to the entire cohort.

Similar to prior LP implementations, concerns with the tool may have overshadowed the benefits of LP. The CodeChat tool was created by one of the authors. The tool is not as polished as commercial or mature open-source software applications. The tool itself works as advertised and is quite stable. Students never reported dissatisfaction with tool functionality. However, installation of the CodeChat tool as problematic. To aid in CodeChat development, the CodeChat tool uses a wide variety of open-source libraries and supporting frameworks. Several of these software packages had varying compatibility issues with student laptops. Students were quite vocal in expressing their frustration with the installation process at the beginning of the semester. The students’ unhappiness was compounded as installation on one student’s laptop would be smooth and flawless, and another student with an identical computer would experience installation problems. Ultimately, several installation problems were a result of students not following the detailed installation instructions provided by the instructor. Eventually, the instructor was able to assist every student and get the tool operational. However, this initial negative experience likely colored the students’ impression of the CodeChat tool and the LP paradigm.

Conclusions

The hope in undertaking our exploratory investigations of LP in hardware-focused courses was that LP would aid student learning, and improve the quality of the student output. We have presented our results with the hope that other educators and researchers will apply LP in microprocessors and HDL education at their institutions to help build a more complete understanding of how LP can support student learning in electrical engineering courses.

We have presented two cases of how LP could be used in hardware-focused courses. In the first case, microprocessors, LP techniques were used by the instructor to present and explain in-class examples. In the second case, digital system design, students used LP principles while writing HDL. Students were less resistant to the new LP paradigm when the instructor was the one intermingling writing and coding. This could be related to our choice to implement LP in advanced hardware courses, which required a minimum of four semesters of programming prerequisite courses in which students did not use LP. In our prior work with LP implementations in introductory courses we have not observed as much resistance and it will be interesting to examine whether students who used LP in early courses have similar or less resistance to LP in advanced courses. However, based on our current data set, we recommend that instructors who implement LP in upper-level hardware courses transition students to the method by first using LP to present and explain in-class examples.

In spite of limitations with the current available tool, students only expressed a slight preference for more traditional pedagogical methods. Additionally, especially for non-computing majors, students recognized that writing about their designs in their natural language could help them express their designs in HDL. This result is promising and motivates further examinations of LP in hardware-focused courses. We challenge instructors to consider ways of implementing natural language writing tasks in conjunction with hardware design activities. While we wait for LP tools to be refined, instructors could consider implementing journaling or reflection prompts into their hardware design assignments.

The efforts reported herein represent only the beginning of the pedagogical possibility for LP in hardware courses. As we continue our parallel work that is exploring how LP and writing-to-learn strategies support students who are at the early stages of learning to program, we will consider how those findings can inform learning gains in upper level hardware courses. Additionally, the results herein support additional research into the effectiveness of instructor use of LP when presenting in-class examples. The results herein also provide evidence to support future investigations into the differences in student perceptions of LP by student background (e.g., novice versus expert, differences by major). From a tool perspective, extending CodeChat from its current state, in which edits can only be made to the source code, to enable direct modification of the LP document, would significantly improve the usability of the system and may assist students in separating their perceptions about the tool from their view of LP benefits.

The post Exploring Literate Programming in Electrical Engineering Courses appeared first on ASEE Computers in Education Journal.

Analysis of Aircraft Actuator Failures within an Undergraduate Experiential Learning Laboratory

Francis Igot — Wed, 02 Dec 2020 13:00:00 +0000

Download PDF

DOI: 10.18260/1-1-110.118.1153-36159

Mario Perhinschi ¹

1 Mechanical and Aerospace Engineering, West Virginia University, Morgantown, West Virginia, 26506-6070, United States

Abstract

The design and implementation of an undergraduate laboratory is presented for the analysis of aircraft actuator failures through simulation. The laboratory was developed in the context of introducing aerospace engineering students to the practical implications of subsystem malfunctions on aircraft dynamics, performance, and control within the general framework of aircraft health management. However, the laboratory assignment can be a valuable addition to other courses in flight dynamics and controls. Advanced simulation tools are used to illustrate concepts and allow students to identify the dynamic fingerprint of aircraft actuator failures and investigate qualitatively their effects on system performance and handling qualities. The computational package relies on Matlab^® and Simulink^®. The typical aircraft aerodynamic control surfaces are targeted in the assignment: aileron, elevator, and rudder. They can be locked at trim or at a different deflection. The main objective of the lab consists in capturing the dynamic fingerprint of actuator failure as a necessary premise for the development of fault tolerant control laws and schemes for subsystem failure detection, identification, and evaluation. The students receive general guidelines on how to design and execute the simulation experiment; however, they are required to answer open-ended questions and encouraged to investigate and follow personal paths in reaching the objective of the assignment. The general design and implementation of the assignment employs active and experiential learning approaches and promotes student initiative and creativity. Instructor direct assessment and student formal feedback demonstrate that active and experiential learning methodologies using numerical simulation are received positively by students and prove to be effective in increasing student motivation and participation and generally enhancing the academic process.

Keywords

Aircraft Health Management, Active and Experiential Learning

Related ASEE Publications

1, 2, 3

Introduction

The design, manufacturing, and operation of modern complex technological products require an integrated interdisciplinary approach referred to as system engineering (NASA, 2007). For an aircraft, a major component is represented by the aircraft health management (AHM), which is aimed at ensuring maximum safe operation within affordability constraints 5, 6. AHM must be considered throughout the entire lifecycle of the system including design, production, operation, and maintenance. The importance of safety for the aerospace industry and research community is expected to continue to grow and, consequently, so does the responsibility of the higher education system to ensure proper workforce background in this area 7. While system operation under nominal design conditions is addressed systematically, operation under abnormal conditions (ACs), when any subsystem malfunctions or encounters off-design situations, is much more difficult to handle, due to the immense diversity of possible scenarios, complexity, multi-dimensionality, and uncertainty.

Academic efforts at West Virginia University (WVU) have been focused on including elements into the aerospace engineering curriculum that are relevant to aircraft operation under abnormal flight conditions 1, 8. Specifically, an undergraduate technical elective course is regularly offered addressing main concepts and important components of AHM 2, 3. It represents a unique attempt in aerospace engineering undergraduate education from the point of view of both the content and its simulation support. As part of this course, a laboratory assignment is implemented focused on identifying the dynamic fingerprint of aircraft actuator failures and investigating qualitatively their effects on system performance and handling qualities.

The lab assignment is supported by an advanced simulation package 2 developed in Matlab^® and Simulink^® that allows user-friendly on-line interaction with students for setting up diverse scenarios including failures affecting major aircraft subsystems. Basic knowledge working with Matlab^® and Simulink^® is necessary for the students to be able to record and process simulation data.

Active and experiential learning approaches9, 10 have been instrumental in designing and implementing the assignment due to their demonstrated capability for increasing the effectiveness of the learning process.

After this brief general introduction, the required student background in flight dynamics and experimental design concepts is summarized in section 2 and 3, respectively. The simulation tools used are briefly described in section 4. The assignment objectives and learning outcomes are outlined in section 5. Laboratory instructions, requirements, and output examples are presented in section 6. A brief discussion on the assessed educational impact and student perception is included in section 7, followed by conclusions in section 8.

Aircraft Dynamics under Actuator Failure Conditions

Students are expected to possess elementary knowledge of aerodynamics and flight dynamics basics, which are typically part of the background at the junior/senior level. They should be familiar with the geometry, role, and functionality of the aerodynamics control surfaces, as well as with the differential equations of motion reflecting the relationships between pilot inputs and the dynamic response of the aircraft 11, 12.

Prior to performing the lab assignment, two to three lectures are dedicated to discussing the general types of malfunctions and damages affecting primary control actuators and their implications on producing aerodynamic control forces and moments. Special attention is given to the jammed control surface scenario, which is the target of the lab assignment.

The primary aerodynamic surfaces, elevator, aileron, and rudder, produce small changes in the lift, which have little contribution to the total lift of the aircraft. However, the moments produced by these small changes are significant and it is these moments that achieve the control of the aircraft. The elevator moves the airplane about its lateral axis, changing the aircraft pitch attitude. The aileron rolls the airplane about its longitudinal axis. The rudder yaws the airplane about its vertical axis. Multiple individual control surfaces are possible on each channel; however, only dual surfaces have been considered for the lab assignment. Note that the aircraft model implemented – a supersonic fighter – features dual vertical tail with left and right rudder.

The jammed control surface scenario assumes that, at a user prescribed moment, the control surface is locked at a certain deflection, current or imposed, and can no longer be moved. Formally, the deflection of an aerodynamic control surface $δ_{c}$ subject to jamming at the current position can be modeled as:

δ_{c} (t) = (\begin{array}{l} \begin{array}{lc} δ_{c} (t) p r o v i d e d b y t h e p i l o t, & i f t < t_{f} \\ δ_{c} (t_{f}) & i f t \geq t_{f} \end{array} \end{array}

where $t_{f}$ is the moment of occurrence of the failure. If the surface is supposed to move to a user specified position $δ_{c f}$ and stay there, then the deflection can be expressed as:

δ_{c} (t) = (\begin{matrix} δ_{c} (t) p r o v i d e d b y t h e p i l o t, i f t < t_{f} \\ δ_{c} (t_{f}) + \frac{1}{τ s + 1} (δ_{c f} - δ_{c} (t_{f})), i f t \geq t_{f} \end{matrix}

where first order dynamics are assumed and the time constant $τ$ must be specified by the user.

Whenever a primary aerodynamic control surface is deflected, a variation of lift $∆ L$ is generated. The effect on drag is negligible. Depending on the location of the aerodynamic center of the control surface with respect to the center of gravity of the aircraft, moments about one, two, or all three axes of the aircraft body reference system of coordinates are produced. The $∆ L$ generated at failure conditions will be different than the $∆ L$ generated at nominal conditions, for the same pilot input. The corresponding moments will also be different. By assessing these differences, actuator failures can be detected and identified.

Elevator failure. Typically, elevators have two symmetric parts and are deflected collectively. Advanced automatic control systems allow for differential deflection as well as a means to increase control redundancy. When one of the elevators is jammed or damaged, the symmetry is perturbed and a coupling between the longitudinal and lateral channels occurs. Let us assume that the full deflection range of the elevator is $[δ_{e m i n}, δ_{e m a x}]$ and the achievable pitch rate range is $[q_{m i n}, q_{m a x}]$ . The positive sign is assumed for the elevator deflected downwards and pitch rate nose-up. With this convention, positive elevator deflection produces negative pitch rate. If one elevator is jammed at $δ_{e} = 0$ , then the pitch rate range becomes $[\frac{q_{m i n}}{2}, \frac{q_{m a x}}{2}]$ . However, if one elevator is jammed at $δ_{e} = δ_{e m a x}$ (corresponding to a deflection saturated downwards), then the pitch rate range is $[q_{m i n}, \frac{q_{m i n} + q_{m a x}}{2}]$ . Because the two elevator surfaces are located on different sides of the plane of symmetry, at post failure conditions, roll rate will be induced whenever a longitudinal command is inputted. A nose-up longitudinal command will induce a roll to the right if the left elevator has failed and a roll to the left if the right elevator has failed. Opposite effects are obtained for a nose-down command. This characteristic can be used to distinguish between a left and right elevator failure.

Aileron failure. Let us assume that the full deflection range is $[{- δ}_{a m a x}, δ_{a m a x}]$ and the achievable roll rate range is $[{- p}_{m a x}, p_{m a x}]$ . The positive sign is assumed for the right aileron deflected downwards and roll rate to the right of the pilot. With this convention, positive aileron deflection produces negative roll rate. If one aileron is jammed at $δ_{a} = 0$ , then the roll rate range is $[- \frac{p_{m a x}}{2}, \frac{p_{m a x}}{2}]$ . However, if one aileron is jammed at $δ_{a} = δ_{a m a x}$ , (corresponding to a right aileron deflection saturated downwards or a left aileron saturated upwards), then the roll rate range is $[{- p}_{m a x}, 0]$ . Note that the effects of a left jammed aileron are very similar to the ones produced by the right aileron jammed at an opposite position.

Rudder failure. If the aircraft is equipped with a single rudder, and the rudder is jammed, then direct control is no longer possible on the directional channel, unless unconventional redundancy is available. If the jamming position is non-zero/non-neutral, then a yaw rate and a roll rate are induced without pilot input. Some compensation can be obtained using ailerons and differential throttle. If the aircraft is equipped with dual rudder and only one of them is jammed, then the overall effectiveness of the directional control is reduced by approximately 50%. The range of achievable yaw rates varies depending on the jamming position. Let’s assume that the moment produced by the rudder is dependent linearly on the deflection angle $δ_{r}$ and that the yaw rate depends linearly on the moment. For healthy symmetric surfaces, the full deflection range is $[{- δ}_{r m a x}, δ_{r m a x}]$ and the achievable yaw rate range is $[{- r}_{m a x}, r_{m a x}]$ . If one rudder is jammed at $δ_{r} = 0$ , then the yaw rate range is $[- \frac{r_{m a x}}{2}, \frac{r_{m a x}}{2}]$ . However, if one rudder is jammed at $δ_{r} = δ_{r m a x}$ , then the yaw rate range is $[{- r}_{m a x}, 0]$ .

Basic Experimental Design Concepts

The laboratory assignment relies heavily on simulation tests performed on desktop computers and in a motion-based 6 degrees-of-freedom (DOF) flight simulator. These tests must be designed and organized carefully to maximize effectiveness within obvious time constraints. As part of the active and experiential learning strategy, the experimental design must be performed by the students. Whether or not they have already been significantly exposed to the concepts and practice of experimental design, a review is necessary, briefly discussing the following topics 13, 14:

main practical objectives of experiments and test for revealing cause/effect relationships;
main phases of the experimental process starting with problem formulation and ending with verification of results;
definition and significance of experimental design components such as factors, levels, and outcomes accompanied by examples on how to use them for flight simulation tests;
full versus fractional factorial design;
performing planned tests, data acquisition, and data sanity verification;
statistical significance;
correlation versus causality;
verification and validation of results and conclusions.

Simulation Tools

Two simulation environments, desktop and motion-based, are used, sharing the same aircraft mathematical model developed in Matlab^® and Simulink^®. Cockpit and scenery visualization is provided by commercially available packages, FlightGear^® and X-Plane^®, respectively. Graphical user interface (GUI) menus allow for the interactive set-up of the simulation scenarios. As an example, the GUI for actuator failure simulation is presented in Figure 1.

Figure 1: Set-up Interface for Aerodynamic Control Surface Failures

The WVU AHM Instruction simulation environment8, 2 is installed on desktop computers and represents the main tool for performing all the simulation tests. Its GUI is presented inFigure 2. The WVU motion-based flight simulator1 , seen inFigure 3, is used to confirm the conclusions of the desktop simulation investigation and provide additional insight into the aircraft dynamic response through more extended sensorial immersion. Due to logistical constraints, only a limited subset of tests can be performed in the motion-based simulator.

Figure 2: Dual Monitor Visual Interface Including FlightGear^® and Simulink^® Visualization

Figure 3: WVU Motion-based Flight Simulator: Instructor Console and Cabin Interior View (left); External View of Motion-based Cabin (right)

Laboratory Objectives and Learning Outcomes

The lab assignment and the supporting lecture segment have been designed aimed at the following main objectives:

review of the most common failure conditions affecting aircraft aerodynamic control surfaces;
analysis of dynamic effects on flight performance produced by actuator failures;
assessment of dynamic signatures of actuator failures through simulation and tests using desktop computer simulation and a motion-based 6 DOF flight simulator;
introduction to experimental design concepts and implementation;introduction to the development of logical schemes for aircraft actuator failure detection, identification, and evaluation;

Upon completion of this lab assignment, the students are expected to achieve balanced levels of complexity and specificity within the cognitive domain according to Bloom’s taxonomy 15. They should be able to:

describe the general dynamic effects of aircraft actuator failures;
design and perform simulation tests on flight simulators, acquire and process pertinent data;
analyze the dynamic signatures of actuator failures and identify their specificity related to the affected element, the type of failure, severity, and interaction with pilot maneuver;
develop simplified detection, identification, and evaluation schemes for aircraft actuator failures;
detect, identify, and evaluate actuator failures based on cockpit instruments and motion perception in the flight simulator.

The assignment design strategy relies on the establishment of mobilizing objectives with the possibility of simplifications applied by students based on their own investigation and identification of critical elements and optimum, most effective path to follow. This makes the laboratory hands-on experience more attractive and even entertaining. Student direct involvement in the decision process for organizing and performing the lab proves to be an effective tool for active and experiential learning. It stimulates student creativity and greatly intensifies their motivation and efficiency.

Laboratory Outline and Instructions

The lab assignment comprises two parts: an investigation based on desktop computer simulations followed by tests performed in the WVU 6-DOF motion-based flight simulator. Due to logistical constraints, only the desktop computer simulation part is extensive and complete. The tests in the motion-based flight simulator are expected to represent a summary of the previous ones allowing for confirming the main results in a more realistic environment including visual and motion cues.

The students are required to design and perform tests in the two simulation environments aimed at capturing and analyzing the dynamic effects of aircraft actuator failure. They are expected to identify the differences in the aircraft responses depending on the failed actuator and the severity of the failure, as a critical premise for developing failure detection, identification, and evaluation schemes. The WVU AHM Instruction simulation package 8, 2 should be used, specifically the model of a supersonic fighter aircraft. The following instructions and minimal requirements are provided to the students:

The analysis must be performed for elevator, aileron, and rudder failures on both the left and right individual surfaces.
A minimum of three different levels of severity must be considered depending on the position of the locked actuator: at current trim, at trim plus a small offset, and at trim plus a large offset. The actual values will be different for each channel and must be determined such that the control of the aircraft can be maintained in all situations.
The main elements of the experiment, namely the factors, levels, and outcome (or response) variables must be clearly identified and justified.
The experimental grid must be kept at a manageable level; however, a minimum of 3 levels must be considered for each factor.
The investigation must be conducted such that all information necessary for detecting, isolating, and evaluating the failure can be obtained. A logical scheme for this purpose must be proposed.
A reduced number of most relevant tests must be selected to be performed in the motion-based flight simulator.
All tests must be performed rigorously as designed. Data recording, labeling, and storage for later use must be performed in a well organized manner.
A flexible Matlab^® script must be built for plotting relevant parameters for analysis and comparison. All graphical and technical aspects of the plots, such as axes labels, title, units, legends, readability, etc. should be properly addressed.
Safety measures and rules must be carefully followed when using the motion-based flight simulator.

The mathematical model of the aircraft is the same for both simulation environments. Up to 3-4 hours are typically scheduled for the desktop computer simulation part. For the motion-based simulator tests, a 20-minute session is dedicated to nominal flight conditions and a 30-minute session, including 10 minutes for each channel is dedicated to failure testing.

The analysis of the dynamic effects of aircraft actuator failures should provide answers to the following questions to be included in the lab report:

What are the main effects of the failures investigated on the dynamic response of the aircraft? In other words, what are the parameters/variables affected, when, and how are they affected?
What are the differences between the failures affecting different actuators?
What is the difference between a failure occurring on the left and right surface in a pair?
What is the effect of increasing the severity of the failure?
What must one do to be able to detect a failure, identify the failed element, and assess the severity? In other words: What are the parameters that need to be looked at? What is the logic of the process? Are there any specific maneuvers or special conditions needed?
What are the differences between the perception of failure dynamic fingerprint in the desktop computer simulation tests and in the motion-based simulator tests?

Laboratory reports are expected to included detailed discussion and comparative analysis supported by relevant plots. Example time histories of relevant states illustrating the effects of different actuator failures on the respective angular rates are presented in Figure 4for elevator failures, in Figure 5 for aileron failures, and in Figure 6for rudder failures.

After the lab, students are required to take a graded quiz/test consisting of three parts. Part I has a homework format and the students are required to produce a detailed plan for a 7-minute test in the motion-based flight simulator during which they would be exposed to an actuator failure affecting an undisclosed control surface starting at an unknown moment. A logical scheme to detect, isolate, and evaluate the failure must be formulated at this time. Part II consists of the 7-minute test in the flight simulator, during which the students have to announce the moment of failure occurrence, identify which of the six control surfaces is affected, and specify qualitatively the level of severity. Finally, Part III represents a take-home post-test analysis, in which the students are required to explain the test based on recorded data and discuss what was done correctly or incorrectly, if their logical scheme was successful or not and why, and what could or should have been done differently.

Figure 4: Effects of Elevator Failures on Pitch Rate

Figure 5: Effects of Aileron Failures on Roll Rate

Figure 6: Effects of Rudder Failures on Yaw Rate

A detailed professionally written lab report is required and counts 11% towards the total course grade, while the 3-part quiz counts 7%.

Academic Impact and Student Feedback

A 10-question multiple-choice quiz was administered before and after the lab to assess its impact in two consecutive semesters with a total enrollment of 35 students. All questions were equally weighted and have been addressed during both the lecture and the lab. Two example questions and their alternative answers are presented next with the correct answers marked in italics.

Example question #1: Consider that the aircraft is initially in a steady-state, horizontal, and symmetric flight. At a certain moment, the left elevator is locked at trim. Without any pilot input, the occurrence of the failure can… (circle one).

a) easily be detected mainly due to the significant pitch rate produced without any other rotation.
b) easily be detected mainly due to the significant roll rate produced without any other rotation.
c) easily be detected mainly due to the significant pitch rate produced coupled with roll rate.
d) easily be detected mainly due to the significant pitch rate produced coupled with yaw rate.
e) not be easily detected.

Example question #2: Consider that the aircraft is initially in a steady-state, horizontal, and symmetric flight. At a certain moment, the left aileron deflects downwards at the maximum possible deflection and remains locked there. The pilot control authority on the lateral (roll) channel is mainly affected as follows: the pilot …(circle one).

a) will continue to be able to command and achieve roughly the same range of roll rates because the right aileron can still be moved over the entire deflection range.
b) will be able to command and achieve roughly half of the range of roll rates in both directions, because only one control surface is now available.
c) can no longer achieve roll rates to the right.
d) can no longer achieve roll rates to the left.
e) can no longer achieve any commanded roll rates.

An average 35% improvement of quiz scores has been recorded after the lab. Three questions that after lecture remained unanswered at a rate of 74%, achieved an average improvement up to 82%. Table 1 summarizes these outcomes.

Table 1: Summary of Lab Effectiveness Reflected by Quiz Scores

	Before Lab	After Lab	Relative Improvement
Total Average Score	67.9%	91.8%	35.1%
3 Lowest-Score Question Avg.	17.1%	82.0%	480%

While the questions included in the Student Evaluation of Instruction were somewhat modified over the years, the average score for the entire course over the past 4 years was 4.77/5. The average of scores addressing student motivation, interest, critical thinking, and thought-provoking level was 4.92/5.00.

Conclusion

An experiential learning laboratory based on simulation for aircraft dynamics under subsystem failure conditions has been successfully designed and taught at WVU.

The academic use of flight simulation tools in conjunction with an open-ended, hands-on assignment has been demonstrated to increase significantly student participation, motivation, and performance.

The implementation of active and experiential learning methodologies based on flight simulation facilitates student initiative and creativity.

Concept understanding, relating cause and effect, and connecting theory and practice in the context of flight dynamics can be more efficiently achieved through this type of laboratory assignments.

The post Analysis of Aircraft Actuator Failures within an Undergraduate Experiential Learning Laboratory appeared first on ASEE Computers in Education Journal.

Simulation and Interactive Digital Tools to Support Teaching Engineering Manufacturing Processes Course

Francis Igot — Tue, 01 Dec 2020 13:00:00 +0000

Download PDF

DOI: 10.18260/1-1-118.1153-36158

Bahaa Ansaf ¹ , Neb Jaksic ¹

1 The Engineering Department, Colorado State University-Peublo, Pueblo, CO, 81001-4901, USA

Abstract

Abstract— Introduction of Manufacturing Processes is one of the core courses in most mechanical engineering, manufacturing engineering, and industrial engineering programs. According to students’ feedback for this course, as well as similar courses offered at different engineering programs, the course is time-intensive, involves no critical thinking, requires limited class participation, and is not well connected with real-world manufacturing problems.

The suggested teaching approach is developed to include several computer-based learning components that can help in creating an active/passive/constructive learning environment for the students. A simulation-based project is used to strengthen constructive concept-based learning and critical thinking for the students and support laboratory analysis. Besides, several online quizzes were developed using a pool of questions related to each topic. Assessments and survey results are used to evaluate the performance of the suggested teaching approach. In addition, students’ micro-lectures are used to improve lifelong learning skills and create an interactive teaching environment with the instructor and other students.

The learning outcomes from the application of the computer aided instruction approach are reflected through the students’ successful completion of the project activities; in addition, the students learned how to use a single computer-aided design (CAD) package to engage in an advanced design and manufacturing analysis which is valued in industry as well as to solve difficult engineering problems. Besides that, the students gained lifelong learning and communication skills through micro-lecture preparations and presentations. In this work, in addition to the students’ performance in the course assignments, a pre- and post-course evaluation is used to asses and evaluate the success in achieving course learning outcomes based on the suggested teaching and learning components. Also, the learning modules related to the casting processes, the product assembly processes, and tolerances analysis topics are presented and discussed.

Keywords

Computer Aided Instruction, Engineering Education, Project Based Learning

Related ASEE Publications

1, 2, 3

Introduction

Instructors are always trying to find a passionate way to teach their courses to support student’s success efficiently and effectively. Furthermore, the continuous increase in the needs for new technical and nontechnical skills in the modern work environment represents another pressure factor on the universities to update student’s learning outcomes to meet the demand of the contemporary industry and business to keep the qualified workers current. Thus, the teaching methods need to be updated continuously to reflect the direct and indirect changes in the learning and the work environment. In general, during the past few decades, engineering education became more focused on hands-on project-based teaching approaches, used more interactive, open-ended problems, and required more feedback on the problem-solving process that is proven to be more effective and can lead to increased student learning 2 .

Several teaching approaches were implemented to improve student’s learning outcomes by integrating active/passive learning and real-life projects. For example, Graham et al. 4 used the Paul-Elder framework of critical thinking to define and operationalize critical thinking for the Electrical and Computer Engineering program students. Students are taught explicitly the methods of critical thinking followed by explicit critical thinking exercises in the introduction to engineering course to prepare students to embrace more elaborate, discipline-specific, critical thinking required of them in future courses. At sophomore, junior and senior levels, courses were selected to emphasize critical thinking and professional ethics. The students were encouraged to use critical thinking skills to analyze requirements and constraints which would apply for advanced real-world problems. Significant improvement in the critical thinking skills of students has been achieved through this sequence.

An integrated thinking approach is adopted by Katz 5 to bridge the educational gap between analytical and design thinking for mechanical engineering students. The suggested approach is implemented by reforming science engineering courses by stressing the physical interpretation of mathematical derivations to analyze and design simple mechanical devices; then, modifying project-based design courses to emphasize the analysis part of the creative design process. Positive feedback from the students suggests that integrated thinking might be successfully applied in many areas of mechanical engineering (ME) education to create continuous education patterns.

Simulation based learning (SBL) provides learners with interactive learning experiences and enhances students’ motivation and performance 6. Their research findings show “that the students perceived their basic psychological needs to be met and that SBL can potentially enhance self‐determined motivation as well as improve learning in general.” Another study 7 shows the value of using simulations to exercise reflective and descriptive thinking of students, given appropriate teacher support and careful technology selection.

Fredriksson 8 explored how and when to use software to support teaching in Engineering, Materials Science, or Design. He described how “a comprehensive digital tool for materials related teaching and learning can be introduced to students and used to promote engineering knowledge, skills and understanding in a modern and accreditation-friendly way.” A software (CES Selector and Granta MI 9) specifically developed for education was introduced in the first-year class on Materials Science and Engineering. The results show that the use of the software contributed to a number of students’ learning outcomes.

A multi-levels sequential design project is used by Ansaf and Jaksic 3 to increase students learning outcomes in design analysis and critical thinking. The students implemented the required design modifications of a product in a systematic time-based procedure using traditional and nontraditional design tools like finite element analysis. Their results show an improvement in student engagement in the course topics and in critical thinking.

Okojie 10 claims that “in a highly competitive manufacturing industry, the total cost of design and manufacturing can be reduced and hence increase the competitiveness of the products if computers can integrate all working procedures. Computer-aided integration has, therefore, become an inevitable trend. Many industries have achieved a great deal of success between non-integrated and integrated systems.”

Egelhoff et al. 11 described “a structured problem-solving approach which uses the students’ understanding of free-body-diagrams, shear and moment equations, and energy methods. With the development of note-taking handouts supplied to the students, the structured analysis is led by the instructor using Castigliano’s theory of internal energy. The problem formulation is kept general until the last step. Numerical integration can be performed in the software of the students’ choice.”; Egelhoff et al. 11 “found that using this approach accomplishes a richer, deeper understanding of design among our students and increases their confidence as indicated by our pre- and post-activity assessment.”

The challenges, as well as the definition, characteristics, and educational objectives of flipped learning are introduced and summarized by Hwang et al. 12. They identify why flipped learning has been adopted by so many educators. Among the reasons presented are:

In-class activities and discussions. This can increase teacher-student and student-student interactions, thus creating an active learning atmosphere that can improve students’ learning motivation.
Multimedia digital teaching materials. These materials “are easy to save, manage, revise, and impart.”
Additional teaching strategies. Advanced teaching strategies capable of promoting higher-order thinking abilities can be implemented.

Wendel 2 used a flipped classroom teaching approach to teach an intermediate undergraduate manufacturing class at the Massachusetts Institute of Technology. According to Wendel, the initial students’ survey indicated that this intermediate-level manufacturing class was not related to “the real world,” was not interesting, and was time-intensive. The feedback from students showed the class to mostly promote informative learning as opposed to concept-based learning and critical thinking. Implementation of the flipped-classroom approach included pre-recorded videos that were used to prepare the students for a lecture. Then students in pairs participated in challenges during the class time related to the lecture topic. The results showed increases in student participation during lecture time. Also, the students noted their preference for advanced scientific content in class.

According to students’ feedback about their learning experience at the engineering manufacturing course, as well as the feedback on similar courses offered at other universities, the students mentioned the following: the course is time-intensive, involves no critical thinking, requires limited class participation, and is not well-connected with real-world manufacturing problems. Part of these drawbacks can be correlated to the traditional course curriculum and teaching style that mainly depend on the lectures for the manufacturing processes that are aligned and synchronized with the laboratory work (projects) to gain the required knowledge and skills.

The preliminary partial results about the design and implementation of a new teaching strategy for this course, and for similar technology-based engineering courses, had been presented elsewhere 1. In this extended work presented here, we address the improvements in the teaching approach of an Engineering of Manufacturing Processes course for mechatronics and industrial engineering students at Colorado State University-Pueblo. The suggested teaching approach is developed to include several learning components that can help create an active/passive/constructive learning environment for the students. Computer-based simulation projects are used to strengthen constructive concept-based learning and critical thinking of the students. Online quizzes are designed and created to help students to improve their understanding of the basic definitions and concepts of traditional and nontraditional manufacturing processes. Additionally, students’ micro-lectures are used to improve lifelong learning skills and create an interactive teaching environment with the instructor and other students.

Assessments and survey results are used to evaluate the performance of the suggested teaching approach during course implementation. In addition, at the end of the course, assessment survey results are used to evaluate the effects of the suggested teaching/learning activities on student learning outcomes.

Course Description

The manufacturing curriculum at the engineering program at Colorado State University- Pueblo consists of a two-course sequence: Engineering of Manufacturing Processes and Computer-Integrated Manufacturing (CIM). Both courses are structured so that enrollment is capped at 24 students per lecture and 16 students per lab section. Both courses are essential for the engineers dealing with any manufacturing discipline, whether working on a factory floor or in a design and management roles. In the Engineering of Manufacturing Processes course, students are exposed to introductory principles and concepts of traditional and nontraditional manufacturing. In general, a manufacturing processes course is a cornerstone foundation course in many engineering programs. The traditional objective of this course is to engage students with the principles and concepts of traditional and nontraditional manufacturing.

The course includes a description and basic analysis of manufacturing processes like product assembly, casting, metal forming, machining, welding, and semiconductor manufacturing. The engineering program at our university offers the Engineering of Manufacturing Processes course with lab (4 semester credit hours) for junior students including 3 contact hours of lecture and 2 contact hours of lab.

Prior to this course, the students had freshman and sophomore level courses and are expected to have the following prerequisites listed by topic:

Basic engineering drawing practices and tolerances
Basic physics concepts: velocity, acceleration, force, torque, energy, power, heat, fluid dynamics
Descriptive statistics, geometry, trigonometry, and calculus
Material properties: strain, stress, strain rate
Graphing 3D objects and system assembly using SolidWorks®,

The number of students in this course is capped at 24. The course is taught only once a year.

Course Implementation

The course is taught by first introducing each topic, then presenting examples, in-class assignments, and projects, and finally assigned homework. The class assignment sets are designed to allow students to practice and sharpen their problem-solving skills. The students are allowed to work in teams to solve in-class assignments during lab time.

In-class Simulation Projects and Exercises

A real-life engineering product with a challenging set of questions is used as an in-class project to improve critical thinking about different manufacturing operations beyond the classroom walls. To accommodate project analysis, the simulation tools in SolidWorks are used. For the dimensions and tolerances analysis in the assembly process, the students work on a design tolerances analysis problem to meet the required design specifications. The tolerances design for a linkage pivot is a modified and extended version from that given by Budynas et al. 13.

Project #1 Tolerances Design for a linkage pivot

A pivot in a linkage has a pin (Figure 1) whose dimension $x \pm a$ is to be established. The thickness of the link clevis is 1.5± 0.005 inches. The designer has concluded that a gap between g_min and g_max will satisfactorily sustain the function of the linkage pivot.

Figure 1: Linkage pivot

For interchangeable assembly processes:

Determine the dimension x and its tolerance a
If the pin diameter available in the stock is 0.6 ±0.002 inch and M1 manufacturing process is used to create the clevis holes suggest an appropriate clevis hole diameter y to ensure the minimum clearance between the pin and the hole is E. (Note: use typical tolerance limits b for M1 process, Table 5.2 (or 5.4) in your textbook)
Use TolAnalyst tool to verify your results from part 1

Notes:

$g_{m i n}, g_{m a x}, E, and M 1$ (b) values are assigned to each student
Show your analysis for the parts A and B precisely using both 100 percent and statistical interchangeability methods.
Submit the tolerance analysis report for the part C in addition to your linkage pivot assembly and the part files.

The expected learning outcomes from this project are:

Understanding the relationship between engineering design of product and assembly process using tolerance analysis as a part of the design specification.
Implementing tolerance analysis for a specific product in x and y-directions using 100% interchangeability and statistical methods.
Understanding the relationship between the tolerance and the type of manufacturing process used to create different features in the components.
Understanding and using the TolAnalyst simulation tool in SolidWorks to implement tolerance analysis for the assembly and compare the results with the traditional methods.

Project #2 Sand casting of a wolf head

This project is the first in a series of engineering manufacturing processes to create a nutcracker as a final project in this class. The first subproject is designed to provide students with a hands-on experience of the sand casting process of a wolf head shown in Figure 2. It is interesting that our engineering department is one of the few that still offer this real experience with the casting process using an in-school foundry. The students need to use design equations for heat and pouring concepts (Equation 1), and the solidification and cooling process analysis (Equation 2 ) 14.

Figure 2: 3D CAD view for the wolf head casting project

The most challenging piece of information that the students need to find is calculating the surface area and the volume of an irregular wolf head part. The students need to use the SolidWorks part file of the wolf head to calculate the surface area and the volume of the casting. Figure 3 shows the casting process in the department’s foundry and Figure 4 shows the final casting product.

Heating and Pouring

H = ρ V [C_{s} (T_{m} - T_{0}) + H_{f} + C_{l} (T_{p} - T_{m})]

H = total heat required to raise the temperature to the pouring temperature [J],
ρ = density [g/cm³],
C_s= weight specific heat for the solid metal [J/(g ^oC)]
T_m = melting temp. of metal in [^oC]
T_o= starting temperature of metal [^oC]
H_f = heat of fusion [J/g]
C_l= weight specific heat for the liquid metal[J/(g ^oC)]
Tp = pouring temperature [^oC]
V = volume of metal heated [cm³]
Total heat required for pouring (H) = ??

Solidification and Cooling (Chvorinov’s rule)

T_{T S} = C_{m} {(\frac{V}{A})}^{n}

T_TS = Total Solidification Time [min]
V = volume of the casting [cm³]
A = surface area of casting [cm²]
C_m = mold constant [min/cm²]
n = an exponent, usually n=2
C_m depends on:
- Mold material
- Thermal properties of the cast metal
- Superheat (pouring temperature relative to the melting point of the metal)
Total solidification time (T_TS) = ??

Figure 3: Sand casting process

Figure 4: Final product

Online Quizzes

The instructor developed a new database that includes more than two hundred questions that cover several topics related to the course materials. The Blackboard® assignment tools were used to create and deliver online quizzes. The online quiz is designed to have 5 -10 questions selected randomly for each student linked to the studied topic in the class. These quizzes are designed to improve the critical thinking and understating of the basic concepts and related terminology outside class time. The duration of the quiz is set to be a one-hour continuous session, and the students are allowed to select the exam time within the given time frame for the exam (usually two days).

Students’ Micro-lectures (MLs)

To create passion and interactive course learning environment students’ micro-lectures for selected topics were introduced and implemented during the class period.

Each student (or a group of students) prepares a 15-20 min presentation to show his/her/their essential findings related to the selected manufacturing process. The micro-lectures focus on the important features and applications of the selected manufacturing process. Video segments and simulations can be used to enrich students’ understanding of that manufacturing process. In addition to the instructor’s evaluation, peer evaluations are used to evaluate micro-lectures. Participation in peer evaluations and discussions is necessary for the final assessment of the micro-lectures. It is expected that the micro-lectures demonstrate essential aspects of the manufacturing process as an added value to the information in the lecture notes. The students are urged to start working on their designated topics when the related chapter is started, as listed in the lecture notes. The micro-lectures weight 15% of the final grade. (75% for the presentation and material quality, 25% for peer evaluation). MLs improve student’s self-learning skills while helping the students achieve the lifelong learning goal. In addition, this learning approach allows more time to focus on problem-based assignments and mini-projects during class time. The students’ micro- lecture topics addressed in this study are listed in Table 1 .

Table 1: Micro-lectures topics

#	Topic	#	Topic
1	Selective assembly	12	Chemical machining
2	Sand casting process	13	Mechanical energy processes
3	Centrifugal casting process	14	Electrochemical machining
4	Vertical casting process	15	Welding
5	Investment casting process	16	Fusion welding
6	Refractory casting process	17	Milling
7	Die casting process	18	Physical vapor deposition
8	Drilling	19	Sheet metalworking
9	Turning and boring	20	Solid-state welding
10	Plating	21	Grinding, honing, lapping
11	Bulk deformation processes	22	Superfinishing polishing, buffing

Results And Discussion

The assessment of the listed outcomes for the new teaching approach is measured directly using the students’ evaluation survey (class participation and critical thinking) and students’ motivation in students’ micro-lectures and projects. The direct evaluation result from class assignments is used to measure a knowledge increase related to the selected course topic. The academic feedback from other faculty members about course implementation strategy and the learning outcomes according to the ABET accreditation criteria is considered in this study as well.

A pre-course survey is prepared to measure students’ expectations of the course in general and the instructor’s teaching style in particular in addition to some questions that are meant to increase students’ awareness of the selected topic (the dimensions and tolerances analysis in the assembly process). The survey results for the class of 24 students showed that about 32% of students hate or dislike the traditional lecture style, 60% do not care, and about 8% love the traditional lecture style.

A post-course survey included more specific questions about the new instructional components in the course (students’ micro-lectures, design-based projects, and online quizzes), in addition to knowledge development assessment questions.

The most interesting result from the post-lecture survey was that 68% of the students answered that they don’t like traditional lecture styles. This shows a strong shift in students’ opinion about lecturing style from the pre-course survey results, and serves as a strong indicator that adding new teaching and learning components makes the class more enjoyable for the students.

In-class Simulation Projects and Exercises

The post-course survey shows that 66.66% of students think that the in-class projects and exercises are very useful and 33.33% consider them as useful. When the students were asked for suggestions to improve the in-class projects and exercises, the most important responses were “help understand material better”, “watch the videos of the applications before doing the exercises”, and “give more time to work on it.” Again time is an essential factor in making sure that the pace of the course does not add more pressure on the students. MLs presentations, including short videos and students’ performances, help students understand the process (virtually) before starting numerical analysis assignments for the in-class projects.

For the in-class Project #1, 75% of the students were able to solve part A successfully, about 67% of the students were able to solve part B successfully, and about 71% of students used simulation tool (TolAnalyst) successfully to verify their results from the traditional tolerance analysis of part A (Figure 5 ). These results aligned with the students’ feedback in the post-lecture survey show that 95 % of the students think that the in-class project helped them enrich their understanding of the class topic. Also, more than 62% of students are willing to use the simulation tools in their future work in industry in addition to 35% that may use it. The post-lecture results show that students think that introducing new simulation tools is very beneficial for their future careers as engineers. This is a good outcome when compared with the pre-lecture survey which shows that about 80% of students in this class did not know the simulation tool TolAnalyst in SolidWorks. Also, about 80% mentioned that working on the in-class project enriches their understanding of the topics. It is interesting to note that adding simulation tools to the project assignment does not require a considerable additional amount of time from the students. According to the post-lecture survey, about half of the students spent 2 hours and about 27% of students spent 4 hours to learn the TolAnalyst tool. Some students struggled with the simulation part of this project due to their lack of basic SolidWorks skills.

Figure 5: Sample of students work for Project #1 part C.

Figure 6: Using Mass properties tool at SolidWorks to find the volume and surface area of the wolf head cast project.

For Project #2 Sand Casting, it was interesting to see that most of the students were able to connect the theoretical casting process analysis with the experimental results for the wolf head casting. Also, they were able to determine the required variables and parameters to complete their analysis and to justify some of the sources of errors using a scientific and engineering approach. Besides, the students understood how essential is a single accurate database from the CAD system to complete the design and manufacturing processes accurately. The students need to use mass properties tools in SolidWorks to find the volume and the surface area of the casting as shown in Figure 6. Both parameters are required to calculate the total heat required to raise the temperature to the pouring temperature and the total solidification time as given in,

In general, students’ notes and comments listed at the project report show an advancement in their fundamental knowledge gain with respect to the casting process, analytical reasoning, and problem solving using available software/hardware tools to connect the process theory with the real casting process. Some students wrote the following in their reports:

“Overall, the experiment was a great learning experience for how casting works in the real world in comparison to the classroom. The equations used in the classroom do not directly transfer to the experiments. Not only did we learn the process but we learned the safety issues and extra steps involved in casting that aren’t taught in the classroom. Also, the lab helped us to understand the work that is behind some of the objects we use in everyday life and how difficult it can be to make them perfectly.”,
“Overall I think this was a very successful lab and I felt as though I could really connect what I was learning in the class to what we were doing in lab. I felt much more comfortable on the test because of my experiences in the lab. I am a very visual learner and the lab has helped me understand the information much more.”
“After having hands-on experience with sand casting, I have a better understanding of the procedures and process needed to make a casting from a mold.”

Online Quizzes

The average quiz score was 92.5%, and all students were able to finish the quiz within the given time frame. The post-course survey results show responses addressing the importance of the online quiz component (not-useful 4.76%, no effect 28.5 %. useful 42.85% and very useful 23.8%). The students suggested the following, “helped reinforce vocabulary,” “yes, improve our definitions.”, and “class quizzes word better.” According to the collected results and students’ feedback, the majority of the students claimed the online quizzes are useful or very useful. However it seems that some students still prefer the in-class traditional paper tests.

Students’ Micro-lectures (MLs)

In general, the student’s feedback about micro-lectures (MLs) was assessed twice, the first time after week 5 of the semester (only assembly and casting processes topics were covered) and the second survey at the end of the semester (after all topics were covered). In the first survey, about 65% liked and about 12% disliked MLs. In the second survey, 95% of the students consider the MLs to be useful or very useful which shows good consistency about the effectiveness of using MLs to promote students’ engagement and improve learning outcomes. In addition, the students were asked to write a single fact they learned from the micro-lectures topic (selective assembly and tolerances analysis). Students’ notes and comments about MLs are in general positive and can be summarized as following: Interesting, improve public speaking and presentation skills, change in class pace and keep students engaged, increase student to student interactions, and makes the students really focus on a subject. Some students disliked the micro lectures for the following reasons: it challenging the students to be expert in this topic, presentation timing, it provides learning opportunity for the student but not for the other students in the class. When the students asked about any suggestion to improve the MLs, the most important responses were: “let the students work in a group”, “keep the timing the same”, and “don’t make them too long”. All positive and negative comments show the importance of improving independent and lifelong learning skills through active learning strategies and class participation and discussions. Also, having a more controlled environment will help in maximizing the ML effectiveness.

The students were asked to write a single fact learned from the micro-lectures topic (selective assembly and tolerances analysis). The student’s feedback shows increased knowledge and advanced thinking about the subject. For the MLs related to the casting topics, the peer evaluation survey for the casting processes listed in Table 1 , shows that more than 80% of the students in the class of 24 students learned or learned much from the micro-lectures. Again, the students were asked to write a fact learned from the MLs topics (sand casting, refractory casting, vertical casting, die casting, investment casting and centrifugal casting). Students’ feedback about what they learned from the MLs show strong evidence that student to student interaction is effective and helpful in learning new facts.

Here are some samples of student’s answers about what facts they learned from students presentations:

“die casting has good dimensional tolerances, centrifugal casting can make giant symmetrical products, vertical casting eliminates trimming.”, “die casting is cost-effective with high volume demand, especially with metals with low melting points.”,
“Investment casting has nothing to do with financial aspects like the name would suggest. It’s actually a wax form casting.”,
“die casting has good dimensional tolerances, centrifugal casting can make giant symmetrical products, vertical casting eliminates trimming.”,
“Centrifugal casting is used in most aircraft, dams, and military products, Die casting is only cost-effective with high volume demand, in vertical casting produce parts with high quality, eliminate trims, quicker molds, reduce cycle time. Refractory anchors are designed to expand to allow the mold to form.”

Figure 7summarizes the effectiveness of different learning activities used in this study (students’ presentations (MLs), in-class projects, and the online quizzes) according to the students’ post semester survey.

Figure 7: Post-course survey results for the implemented learning activities

CONCLUSIONS

In this work, a computer-based learning and digital resources (simulation-based projects and online quizzes), in addition to the students’ micro-lectures were used to create an active/passive/constructive learning environment to support the teaching of the Engineering of Manufacturing Processes course. Simulation-based projects were used to support constructive concept-based learning and critical thinking for the students and to integrate simulation analysis with real manufacturing processes in the lab. Online quizzes were used successfully to help students practice and understand the basic concepts and related terminology outside class time. Students’ micro-lectures supported the development of lifelong learning skills and created an interactive teaching environment with the instructor and other students. In general, the essential lessons and outcomes from this study can be summarized as follows: First, the simulation-based design projects helped in enriching students’ understanding of the studied topic and improved their ability to address the real-world problems and analysis and use their engineering judgment to draw conclusions (ABET Criterion 3. Student Outcomes: 6). Second, introducing simulation tools as a part of the learning environment can be implemented easily and without burdening the students much, especially if they already used the same CAD software as a drafting and design tool. A single CAD database can be used to produce many types of drawings and models used throughout the design and manufacturing processes. Third, online quizzes can be used to help expose students to new terminology and definitions, as well as to learn on their own, based on the available online resources and the textbook. Forth, using students’ micro-lectures helped in improving students’ life-long learning and communication skills (ABET Criterion 3. Student Outcomes: 3 and 4). Fifth, students’ micro-lectures increased students’ learning outcomes by making the class more interactive, and eventually, can be used to expose the students to the manufacturing process concepts and methods that can help them solve numerically-based exercises and problems.

The suggested teaching strategy can work effectively with small classes and maybe mid-sized classes if the instructor (s) can provide adequate resources. For future work, the authors are planning to introduce more simulation-based projects in the curriculum like machining simulation and cost analysis of casting.

The post Simulation and Interactive Digital Tools to Support Teaching Engineering Manufacturing Processes Course appeared first on ASEE Computers in Education Journal.

Promoting STEM to Middle School Girls through Coding and Fashion

Francis Igot — Wed, 01 Jan 2020 22:19:32 +0000

Barbara L. Stewart

Professor, Human Development and Consumer Science
University of Houston
bstewart@uh.edu

Carole Goodson

Professor, Technology
University of Houston
cgoodson@uh.edu

Susan L. Miertschin

Associate Professor, Teaching, Computer Information Systems
University of Houston
smiertsch@uh.edu

Susan L. Schroeder

Lecturer, Program Manager
University of Houston
SSchroeder@uh.edu

Misha Chakraborty

Adjunct Professor
University of Houston
Prairie View A&M University

Marcella Norwood

Associate Professor, Program Coordinator, Global Retailing M. S.
University of Houston

Abstract—Informal education approaches such as science camps have been used to promote interest in STEM fields for pre- college students. This paper presents the evaluation of outcomes of a computing outreach initiative targeting middle school aged girls from populations currently underrepresented in STEM. A fashion and retail themed code camp was offered free of charge through a grant from the Texas Workforce Commission and was assessed to be highly successful with respect to student and parent satisfaction. Assessment for changes in attitudes toward STEM and coding also showed positive changes.

Keywords—coding camps, STEM, middle school education, STEM education, workforce preparation, career preparation

I. INTRODUCTION

DesignHER Code Camps were created to intrigue middle school aged girls with technology focused career opportunities in the familiar and engaging world of retail and fashion. The retail and fashion industry was chosen to highlight because, while many middle school aged girls may be keenly interested in fashion and clothes, the researchers believe fewer would view fashion careers as being related to or requiring knowledge and study of science, technology, engineering and mathematics (STEM). The camps were envisioned as tuition-free, fun summer experiences with the purpose of drawing girls from populations underrepresented in STEM fields toward STEM preparation. The camp curriculum was developed to showcase coding and programming applications related to retail and fashion, fields to which, based on the authors’ experience, many female middle schoolers seem to be already drawn through interest and accessibility.

Technology is embedded in all aspects of the retail and fashion industry, from online ordering and marketing, to inventory control and human resource operations. Professionals in this field need to be knowledgeable in STEM applications as well as the retail and fashion field. Modern mechanisms of fashion and retail are information technology dependent and employment demands for technology workers to drive the industry’s operations are high. The National Retail Federation’s (NRF) online publication Stores, in their cover story “What’s in store for 2018?” reported the biggest impacts emerging for that year. Almost every impact presented was dependent on coding- driven technologies. Coding dependent impacts presented included augmented reality, artificial intelligence, voice user interface empowered by natural language processing, cybersecurity monitoring, and Internet of Things [1]. It was also reported in a NRF blog post that retail openings for computer science graduates numbered roughly 5000 in 2017, which would constitute the retail sector hiring 15% of all computer science graduates in that year should all the positions be filled [2]. The great need for STEM workers in retail and fashion supports the choice of the industry for the camp focus.

The University of Houston, aided through a grant from Texas Workforce Commission, provided DesignHER Code Camp experiences for a few more than 130 middle school aged girls during the summer of 2019. Each camp engaged students in hands-on coding and career experiences that provided challenging and innovative concepts in computational thinking, problem-solving, and analytical skills, while fostering an interest in computer coding and programming. Workforce readiness and career preparation was encouraged by exposing the girls to careers with a focus on retail and fashion, which are typical areas of interest during the middle school years and are areas that require STEM skills. As designed by faculty at the University of Houston, the DesignHER Code Camp served two primary goals.

Goal: Increase the interest of middle school girls in coding and programming.
Goal: Provide hands-on experiences in summer camps that include challenging and innovative concepts and experiences in computational thinking, problem solving, and analytical skills, and simultaneously foster an interest in Science, Technology, Engineering, and Math (STEM) related careers.

The purpose of this paper is to present the University of Houston’s experience in offering a fashion themed code camp. It outlines how such a camp can be conducted. In addition, evaluation results for impact on the participants are presented. The authors hope the paper suggests a model for future camp experiences that might be offered to promote STEM to pre- college students.

II. REVIEW OF LITERATURE

A. Demand for STEM Skills

Dynamic growth of STEM related jobs continues to create demand for job-ready workers [3]. Skills for job-readiness include STEM knowledge as well as communication, critical thinking, and collaboration skills [4,5]. The need for STEM skills in the workplace spans all industries today. As industries turn to artificial intelligence and data science in support of decision-making processes, all remain or become industries in need of STEM workers, especially in areas related to computer and information science and coding. Employers across all industries struggle to find enough skilled workers in STEM fields [6], thus, attracting young women who are underrepresented in STEM work is important.

Seeking a solution to supply future demand for STEM workers, Fields recommended camps utilizing an informal science education approach as a means to increase enthusiasm and interest towards science education [7]. The suggestion is consistent with the position of the National Science Teachers Association that summer science camps can be valuable for promoting STEM to youth by emphasizing creativity and enrichment, developing intellectual curiosity and sharing it as modeled by appropriate and appealing camp mentors, and exploring science careers through interaction with role models from the professional community [8].

B. Benefits and Effects of Camps

Fields’ recommendation to use informal education approaches such as camps [7] is supported by research. For example, Foster and Shiel-Rolle found through a pilot study that low cost science camps are feasible solutions to bring relevant science focused information to students and that the STEM camps had positive impacts on science literacy and career goals related to science subjects [9]. Research reviewed revealed multiple and varied positive impacts of summer STEM camps.

Attitudinal changes in student participants was a positive outcome from some camp research. For example, Ouyang and Hayden examined the attitudes of middle school Hispanic students toward science and technology following a STEM camp for 7th and 8th graders, and found positive outcomes [10]. Fifth and six grade students benefited attitudinally in a study conducted by Bathgate, Schunn and Correnti [11]. Attitudinal factors showing improvement among participants included understanding the value of science (appreciation), excitement in learning about science (curiosity), recognition of the role they themselves can play in ‘doing’ science (identity), confidence in their own ability to do science (self-efficacy), developing sufficient interest in science to motivate learning it (motivation), and the ability to remain focused even when facing obstacles (persistence). Two hundred and fifty-two fifth grade and sixth grade students participated in the study, and a key finding was that students’ motivation is an important factor for camp success. Dalbotten et al. measured success from student participation in secondary level astronomy camps over four years [12]. The researchers measured seven factors: see, quantify, describe, tinker, grow, relate, and understand. They observed a significant amount of growth in a multi-factor attitudinal construct termed self-directedness for camp participants who attended consistently for four years. Studies such as these demonstrate that camp experiences can change student attitudes.

Results from a number of STEM camp studies focused more on content outcomes with respect to the questions (1) did the participants learn STEM concepts and (2) is the content viable for learning by younger students. A study already mentioned found improvements in science literacy concepts among participants [9].To address the need to increase student interest in computer engineering, Clark and Pais conducted a Scratch (programming) summer camp for elementary level children [13]. Small group activities were developed and used to teach programming with the block-based Scratch programming ‘language’. Camp teachers enthusiastically affirmed the possibility of teaching elementary children programming concepts through Scratch. Hammak, Ivey, Utley and High’s research concluded that students gained improved understanding about engineering subjects after the STEM camp they conducted [14].

Another aspect of the literature focused on elements and practices that were proven effective for camp success. Fields found that peer relations and peer interactions contributed toward a positive camp environment for an astronomy focused camp [7]. In addition, the researcher found that students felt empowered when they autonomously used the equipment and technology. Further, learning from peers and seeing and experiencing how to use learned concepts and knowledge in other contexts emerged as important factors for camp success. Collaboration between engineers and K12 teachers in building and delivering camp content was identified as an important success factor for science camps and this finding endorsed the importance of partnerships between teachers and engineers or other practitioners in designing and delivering camp content [14]. Ouyang & Hayden concluded that camp teachers were an important element of success, noting that “with proper support, science teachers can become advocates for both computing concepts and technology related careers” (p. 233) [15]. Research by Elam, Donham, and Solomon concluded that collaborative activities and interaction with career professionals was the most valuable to participants [16]. Another study by Yildirim and Türk captured some specific teacher perspectives about STEM camp success including the following observations [17].

Camp assignments should have a real-life orientation.
Camp teachers should be competent in the STEM content knowledge that is the focus of the camp.
Camp teachers should demonstrate an understanding of pedagogical principles and practices, integration knowledge, and 21st century skill knowledge.
Content must be age appropriate.
Camp teachers must be able to establish positive relationships with the students.
Time and classroom management skills are of high importance.

Some of the research indicated areas in need of further study. Fields concluded that further investigation is needed to understand the impact of a short-term science camp on youths’ long-term identification with and possible pursuit of a career in STEM [7].

III. CAMP DESIGN AND DEVELOPMENT

The College of Technology at the University of Houston secured grant funding from a state agency to support offering a retail and fashion focused code camp. The use of fashion as a link to interest in STEM fields, while a novel approach, was also used by Ogle, Hyllegard, Rambo-Hernandez, and Park [18].

In partnership with the Houston Independent School District and charter schools in Houston lower income areas, the grant enabled provision of three week-long academic DesignHER Code Camps for girls in sixth through eighth grades, especially seeking to serve students from low-income families and minority students underrepresented in STEM fields, and including youth in foster care as well as students with disabilities.

Three consecutive weeks of camp for students entering grades 6 through 8 were conducted, with each week focused on girls in one of the three particular grade levels. Activities focused on coding and computational thinking related to fashion and retail. Among the vast array of areas requiring STEM skills, retail fashion careers that require coding and programming skills had the spotlight in support of increasing demand for information technology and computer science graduates in that industry [1,2]. The authors’ personal experience indicated that many middle school girls are interested in fashion and clothes, yet, fewer understand how the industry relates to STEM careers. Campers gained exposure to opportunities to develop and apply coding knowledge to careers related to something they already love.

Camp activities created to foster interest in STEM careers in retail included individualized coding activities, group activities with a fashion theme, casual interaction with women employed in STEM jobs, field trips to retail and fashion related venues, and interaction with camp counselors who were young women poised to enter the STEM workforce, some of whom were pursuing majors in retail, computer information systems, and computer science. Workforce readiness and career preparation were encouraged by exposing the students to careers within areas of interest during the middle school years and that require STEM skills.

A first-day activity for each camp week was a video that highlighted national award-winning STEM projects completed by youth. The purpose was to inspire campers to believe that they, too, can excel in STEM projects by seeing peer-group aged individuals who had done so.

For coding activity and instruction, hands-on experiences from Google sponsored CS First — Fashion and Design Curriculum [19] and from Hour of Code [20] were selected. DesignHER project investigators chose the Fashion and Design curriculum from CS First because (1) the content area was on target with the camp purpose and (2) a complete curriculum was provided that included video instruction, hands-on activities, and even marketing materials. Further, the CS First material had been thoroughly tested and improved based on feedback from camps and classrooms nationwide (possibly, worldwide). The instructional staff field-tested the CS First curriculum in advance and found it very easy to follow and found that the assignments would engage middle school age girls. Hour of Code activities were selected to fill in other camp laboratory hours with hands-on activities based, usually, on apps built by supporters of the Hour of Code project to illustrate and reinforce a focused concept or construct related to coding and/or, more broadly, to computational thinking.

Camp staff tested and selected Hour of Code activities and incorporated these as needed to fill in hour-long time blocks between other activities. Each camp week focused on a different student grade-level, so more Hour of Code activities were planned (and needed) for rising eighth graders than sixth graders. Camp staff had outside prior knowledge of several online code teaching tools with a free level of use such as Codecademy (https://www.codecademy.com) and CodePen (https://codepen.io) that proved to be useful to supplement the planned coding activities as needed.

A number of planned activities took the campers outside of the computer lab to other areas of campus to show them some of college life. On-campus activities included in each week-long camp were a campus wide scavenger hunt, time spent with technology professors, and a 3D printing demonstration (to highlight broader STEM areas also poised to have an impact on retail [1,2]). The goal of these on-campus activities was to encourage campers to visualize their future selves as individuals pursuing higher education by attending college.

Group activities were planned in order to promote bonding among campers and between campers and camp counselors and teachers. While many of these activities were planned, a fashion show where student groups designed and created garments from toilet paper was a highlight.

Some activities each week focused on STEM and coding related careers and broadening the students’ ability to see themselves in a STEM or a coding related career. Women employed in STEM fields were recruited to serve as role models and they were scheduled to interact with campers during lunch sessions. In addition, two field trips focused on the information technology behind retail were scheduled each week where students interacted with professionals at their places of employment, which were retail locations of both Target and Buckle. Students observed retail use of digital devices used to control inventory, an online ordering system, and an employee scheduling system. Professionals spoke to the importance of these technologies to retailers and the need for workers who understand the use and the development of such technologies that depend heavily on the coding that underlies the function. Posters were created and used to highlight the contributions of women in STEM careers. These were used for a day of camp each week that was themed as “Women Who Work”.

In order to offer a successful camp that also met all organizational requirements for working with youth to assure their safety and privacy, much consideration had to be given to camp logistics during the semester prior to offering the camp. Consideration spanned the concern areas of safety and health, meals and snacks, safe and within-budget bus transportation to and from campus to two field trip locations each week, safe and within-budget bus transportation to and from the original partner schools to camp each day (which was deemed necessary for this effort since it targeted lower socio-economic populations who may be challenged by transportation need), faculty and staff hiring and assignment, campus physical arrangements on the UH campus such as computer labs with software needed for the activities, logins for the computers for the student guests, computer security with respect to online vulnerabilities for youth, and the processes for reaching out to the community to recruit campers, certify their eligibility, and enroll them in camp.

To promote the camps to the targeted demographic population, meetings were held with the principals of four original partner schools and one social services agency in the Third Ward area in Houston. The Third Ward area lies in close proximity to the University of Houston where the camp would be hosted, and it offers ample opportunity to serve students traditionally underrepresented in STEM fields and traditionally underserved by developmental STEM camps. To encourage greater participation of girls in foster care, a social service agency was added to the school partners. Each partner school or agency recruited camp participants and distributed and collected applications from individuals.

As the original application due date approached, the recruitment and outreach effort expanded to include more schools in the Houston Independent School District. Outreach to these additional partners was designed to increase enrollment from the desired demographic; but transportation to and from camp would not be able to be provided for any girl attending from outside the Third Ward. In two cases, camp faculty were invited to parent meetings at public schools to discuss and promote the camps. The recruitment effort was successful and camp enrollment was closed at 132 campers. Additional qualified sixth and seventh grade student applicants were placed on wait lists for the first two camp weeks.

IV. CAMP ASSESSMENT AND RESULTS

The eligible population served consisted of female middle school students who completed the fifth grade by the first day of camp but who had not yet completed the eighth grade by the last day of camp. The population was divided into same-grade cohorts (rising 6th, 7th, and 8th graders) with each week of camp serving a single grade-based cohort. Documentation for each participant regarding grade level completed was reviewed after the application deadline and verified prior to notification of student acceptance.

Program assessment was accomplished using several measurement instruments and techniques. These were pre- and post-camp student questionnaires, camper and parent written comments, and post-camp focus groups with camp staff and faculty. Camper pre- and post-camp data were collected via a 66-item questionnaire completed by the campers during the morning of the first camp day prior to any coding activities and mid-day of the last camp day. The survey instrument was adapted from the “Middle School Attitudes toward STEM” instrument developed and validated by a team of researchers with the William and Ida Friday Institute for Educational Innovation [21].

Both camper and parent comments were abundant. Some comments were formally solicited and some were unsolicited. Unsolicited comments from parents were most often received in response to informational emails sent by the camp director to keep parents and families informed of the camp’s progress and next-day activities. Campers provided comments on an open- access physical poster available on site which was dedicated and made available to them for the specific purpose of providing feedback of any sort. Finally, all project staff, including the instructional staff and the principle investigator for the grant engaged in a focus group discussion at the conclusion of the project.

Pre-camp and post-camp assessment results provide evidence of the success of the DesignHER code camp project. The findings represent camper and parent perceptions of value attributed to and satisfaction with camp elements and activities.

Post-camp survey summary data are presented in Table 1. The data show that most campers felt they both learned from and enjoyed their DesignHER Code Camp experiences. In fact, by collapsing the “agree” and “strongly agree” response categories the following positive results are evident:

100% of 8th graders reported that they learned from the coding activities.
Similarly, 94.8% (6th grade) and 95.6% (7th grade) of participants learned from the coding activities.
97.5% (6th grade), 93.3% (7th grade), and 95.2% (8th grade) of participants enjoyed the camp experience.
89.8% (6th grade), 88.9% (7th grade), and 80% (8th grade) of participants learned from the field trips.
87.2% (6th grade), 88.9% (7th grade), and 85% (8th grade) of participants learned from the group activities.
89.7% (6th grade), 93.4% (7th grade), and 82.5% (8th grade) of participants learned from meeting the experts (role model professional women employed in STEM).

Table 1. Response Distribution (%) by Camp Experience Item and Grade Level

The evidence shows that the DesignHER Code Camp was highly successful in providing an enjoyable and productive learning experience for the girls.

Post-camp and pre-camp assessment data show attitudinal positions of camp participants with respect to the activity of coding and the importance of coding in the workplace. The collected data and the analysis reveals a strictly positive change in attitudinal position with respect to the activity of coding for the majority of camp participants. For example, post-camp and pre-camp assessment data indicate that the girls feel sure of themselves when doing coding. At the conclusion of each camp week, campers agreed or strongly agreed with the statement, “I am sure of myself when I do coding.” The expression of agreement with the statement was at the levels of 87% (6th grade), 77% (7th grade), and 73% (8th grade). These numbers represent a successful impact by the DesignHER camp to the extent that they are positive changes across grade levels in the range of 8% to 44% over pre-camp assessment based on the same item.

Similarly, pre- and post-camp assessment data showing attitudinal positions of camp participants with respect to consideration of coding as a career and its importance in the workplace showed similar strictly positive changes across grade levels. Notably, a substantial number of the campers reported that they would consider a career in coding. Camp participants agreed or strongly agreed with the statement, “I would consider a career in coding” at the rates of 41% of 6th graders, 53% of 7th graders, and 48% of 8th graders. The post-camp assessment summary of responses to the item represents a positive change in the range of from 15% to 18% for attitudinal position on this construct.

For all of the statements in Table 2, student responses indicated positive change in post- over pre-camp survey responses with respect to confidence in coding ability, interest in coding, and awareness of career connection to coding.

Table 2. Response Distribution (%) by Item/Combined Grade Levels

Both unsolicited and solicited comments regarding the camp experience were collected and compiled. Unsolicited, parents sent email messages to the camp director to share the camp success stories their girls were sharing at home. Much like the mother who said, “My daughter had never coded before, and now she wants a career in coding”, family comments focused on coding and general growth as well as specific experiences of the camps. The success stories of the camps are shared in the words of the campers and their families in Table 3 Parent Comments – Unsolicited and Table 4 Student Comments – Solicited.

Table 3 – Parent Comments – Unsolicited

Tabe 4. Student Comments – Solicited

At the conclusion of the three camp weeks the faculty and staff participated in a focus group session to evaluate camp strengths and weaknesses. The focus group assessment yielded overwhelmingly positive results. The DesignHER camp faculty and staff found the positive, fast-paced, productive environments of the camps to be energizing and very rewarding.

Items reported as most positive by the faculty and staff included:

Individual interactions – among campers, mentors, and staff
Exposure to a variety of careers – STEM and others
Connections between coding and fashion and retail as well as other careers
Combination of coding and retailing
Connections to parents via the director’s emails and campers’ comments
Making new friends with campers from other schools
No division among girls
Flexibility with planning – a “can do” attitude
Field trips – both leaders and campers learned that coding is pervasive across industries, including the retailers they visited. Retail coding applications became a “stepping-stone” to seeing coding uses in other fields.

Items reported as needing improvement or deletion by the faculty and staff included:

Registration process – an electronic registration process was discussed as desirable.
Application and promotion materials need to be in Spanish as well as English.
Bus transportation from partner schools to and from campus for camp (not for field trips) was discussed as not critical.

CONCLUSIONS

In summary, the results of the pre- and post-camp assessments, the summarization and analysis of camper and parent comments, and the feedback from the post-camp staff and faculty focus group collectively illustrate the substantial success of the DesignHER Code Camp in providing fun developmental learning experiences for the group of middle school girls served.

Activities supported as beneficial to campers were numerous and included field trips to Target and Buckle to see behind-the-scenes technology aspects of retail, a UH campus scavenger hunt, hands-on coding exercises and interactive games, interaction between campers and professional women employed in STEM fields, and the safe camp environment that eliminated competition and comparison of individuals. Benefits accrued to the campers included the girls acquiring an expanded view of STEM careers and in their future role in this field. There were indications that the girls became more self-confident in their coding abilities. Additionally, the girls engaged in an on-campus, university readiness experience to encourage education beyond middle and high school.

Four camp areas presented more challenges than anticipated. One was the enrollment process which proved much more time consuming than expected. More time was required to communicate with school and agency personnel; to respond to questions from schools, agencies, and parents; to distribute and collect applications; to review and process applications; and to communicate with parents regarding incomplete information and individual needs. Another challenge was found in strict organizational and state protocols for contracting for goods and services in support of camp activities. A third problem arose from the success of recruiting campers underrepresented in STEM which included Hispanic students. The need for Spanish language translation was not anticipated. The camps were conducted in English without issue, but many parents spoke only Spanish. Translation was needed for flyers, posters, application materials, and emailed communications. The fourth major challenge was the unexpected fluidity of camp enrollment prior to the first day of camp. A significant number of campers declined the opportunity to attend after accepted for a variety of reasons. Fortunately, wait listed students were admitted and filled those openings.

Early success stories centered on the strong enthusiasm for the DesignHER Code Camps among the camp school and agency partners. One school created a “reveal” event to notify their students of the camp opportunity. An evening was chosen, the principal selected students she knew could benefit and who would likely be honored by inclusion, printed invitations were given to the girls and their parents, the room was decorated with balloons denoting a special event, refreshments were served, and the PI and Camp Director were invited to “reveal” the camp opportunities. It was great promotional event for all who attended. Further, the University of Houston showed enthusiasm for the DesignHER Code Camps by highlighting the camps in official publications. Retailers were eager to support the camps because of their focus on retail technology and beyond. Commercial retail sponsorship was also exceptional.

The experience shared here illustrates a successful summer camp designed to increase interest in STEM and STEM careers for middle school girls, especially in the area of coding. It can serve as an example for educators seeking means to encourage and extend interest in STEM skills and careers among various populations.

REFERENCES

[1] S. Reda. “What’s in store for 2018?” Stores, 04 Dec 2017. Retrieved Dec 2019 from: https://stores.org/2017/12/04/looking- forward/?utm_source=RetailRecap&utm_medium=12- 10&utm_content=What’s-in-store-2018&utm_campaign=SmartBrief.

[2] T. Ryan. “Can retail compete for computer science graduates?” 08 Aug 2018. Retrieved Dec 2019 from: https://retailwire.com/discussion/can- retail-compete-for-computer-science-graduates/.

[3] Partnership for 21st Century Skills, Homepage, 2017. Retrieved May 2019 from: http://www.21stcenturyskills.org/index.php.

[4] G. Naizer, M.J. Hawthorne, and T. Henley. “Narrowing the Gender Gap: Enduring Changes in Middle School Students’ Attitude Toward Math, Science and Technology,” Journal of STEM Education: Innovation and Research, 15(3), 2014, pp. 29-34.

[5] The Competitiveness and Innovative Capacity of the United States. Washington, DC: United States Department of Commerce, 2012.

[6] P. Mikalef and J. Krogstie. “Big Data Governance and Dynamic Capabilities: The Moderating effect of Environmental Uncertainty,” PACIS 2018 Proceedings, 2018, p. 206, https://aisel.aisnet.org/pacis2018/206.

[7] D.A. Fields. “What do Students Gain from a Week at Science Camp? Youth perceptions and the design of an immersive, research-oriented astronomy camp,” International Journal of Science Education, 31(2), 2009, pp. 151-171. DOI: 10.1080/09500690701648291.

[8] “Informal science education: Position statement of the National Science Teacher Association,” Journal of College Science Teaching, 28, 1998, pp. 17–18.

[9] J.S. Foster and N. Shiel-Rolle. “Building Scientific Literacy through Summer Science Camps: A Strategy for Design, Implementation and Assessment,” Science Education International, 22(2), 2011, pp. 85-98.

[10] L.S. Nadelson and J.M. Callahan. “A Comparison of Two Engineering Outreach Programs fo Adolescents,” Journal of STEM Education: Innovations and Research, v12 n1-2, Jan-Mar 2011, pp. 43-54.

[11] M.E. Bathgate, C.D. Schunn, and R. Correnti. “Children’s motivation toward science across contexts, manner of interaction, and topic,” Science Education, 98(2), 2014, pp. 189-215.

[12] D. Dalbotten, E. Ito, A. Myrbo, H. Pellerin, L. Greensky, T. Howes, and C. Kowalczak. “NSF-OEDG Manoomin Science Camp Project: A model for engaging American Indian students in science, technology, engineering, and mathematics,” Journal of Geoscience Education, 62(2), 2014, pp. 227-243.

[13] J. Clark, M.P. Rogers, C. Spradling, and J. Pais. “What, no canoes? Lessons learned while hosting a scratch summer camp,” Journal of Computing Sciences in Colleges, 28(5), 2013, pp. 204-210.

[14] R. Hammack, T.A. Ivey, J. Utley, and K.A. High. “Effect of an Engineering Camp on Students’ Perceptions of Engineering and Technology,” Journal of Pre-College Engineering Education Research (J-PEER): Vol. 5: Iss. 2, Article 2, 2015. https://doi.org/10.7771/2157- 9288.1102

[15] Y. Ouyang and K. Hayden. “A technology infused science summer camp to prepare student leaders in 8th grade classrooms,” Proceedings of the 41st ACM technical symposium on Computer science education, Mar 2010, pp. 229-233.

[16] M.E. Elam, B.L. Donham, and S.R. Solomon. “An Engineering Summer Program for Underrepresented Students from Rural School Districts,” Journal of STEM Education: Innovations & Research, 13(2), 2012, pp. 35-44.

[17] B. Yildirim and C. Türk. “Opinions of Secondary School Science and Mathematics Teachers on STEM Education,” World Journal on Educational Technology: Current Issues, 10(1), 2018, pp. 52-60.

[18] J.P. Ogle, K.H. Hyllegard, K. Rambo-Hernandez, and J. Park. “Building Middle School Girls’ Self-Efficacy, Knowledge, and Interest in Math and Science Through the Integration of Fashion and STEM,” Journal of Family and Consumer Sciences, 109(4), 2017, pp. 33-40.

[19] CS First Fashion and Design Curriculum. See https://csfirst.withgoogle.com/c/cs-first/en/fashion-and–design/overview.html.

[20] Hour of Code Activities. See https://hourofcode.com/us.

[21] Middle/High School Student Attitudes toward STEM Survey, Wiliam and Ida Friday Institute for Educational Innovation, Raleigh, NC, 2012. See https://www.fi.ncsu.edu/.

BIOGRAPHICAL INFORMATION

Barbara L. Stewart

Professor, Human Development and Consumer Science
University of Houston

Barbara L. Stewart earned a B.A. from Brigham Young University, an M.S. from Utah State University, and an Ed.D. from Brigham Young University. Her research and curriculum development interests focus on online course development and delivery: and cognitive, multiple talent, and learning styles theories and their application to educational settings. Dr. Stewart’s career has included service as a faculty member, department chair, and associate dean. She is currently professor of Human Development and Consumer Science at the University of Houston.

Carole Goodson

Professor, Technology
University of Houston

Carole Goodson is a Professor of Technology at the University of Houston. As an active member of ASEE, she is a member of the Academy of Fellows, a past Editor of the Journal of Engineering Technology, a past Chair of PIC IV and the ERM Division, and a past Chair of the Gulf Southwest Section of ASEE.

Susan L. Miertschin

Associate Professor, Teaching, Computer Information Systems
University of Houston

Susan L. Miertschin is an Associate Professor teaching in the Computer Information Systems program at the University of Houston. Her teaching interests are in the development of information systems applications and the complementary nature of back-end developer and front-end developer skill sets. Her research interests are program and student assessment, the impact of instructional technology on student learning, and the improvement of e-learning environments and experiences. She earned B.S. and M. Ed. degrees from University of Houston, and an M.S.I.S. from Dakota State University.

Susan L. Schroeder

Lecturer, Program Manager
University of Houston

Susan L. Schroeder is a Lecturer and Program Manager, teaching Discrete methods in technology, applied numerical methods, applied statistics, calculus, and trigonometry for the College of Technology at University of Houston. She earned B.S. in Secondary Mathematics Education with a Specialization in Computer Science from Lock Haven University of Pennsylvania and a M.S. degree in Mathematics from the University of Houston at Clear Lake. She has 34 years of experience teaching mathematics and computer science, with the last 26 years at the University of Houston. She served as Camp Director for the DesignHER camp.

Misha Chakraborty

Adjunct Professor
University of Houston
Prairie View A&M University

Misha Chakraborty is completing a postdoctoral assignment with the Office of Professional and Graduate Studies at Texas A & M University. She completed her Ph. D. in Human Resource Development there. She has served as a director of STEM programs for elementary through pre-college students and she presently serves as a board member of Auyr Inspiration, a company that provides programs to teach young children art and music. She is an adjunct professor at University of Houston and at Prairie View A & M University. She has published papers in reputed journals and attended many conferences. She is in quest of spreading knowledge among children and adults. She is associated with various non-profit organizations that work towards education and safety of young children in developing countries.

Marcella Norwood

Associate Professor, Program Coordinator, Global Retailing M. S.
University of Houston

Marcella Norwood, Associate Professor and Program Coordinator for the Global Retailing M.S. at University of Houston, earned degrees from San Jose State University (B.S.in Marketing, M.Ed. Business Education), Colorado State University (M.Ed.), and Auburn University (Ed.D). She received the Outstanding Teaching Award from the University of Houston (UH), and Outstanding Teaching and Faculty Service Awards from the UH College of Technology. She serves as Business Education Forum Marketing Editor, DECA International Career Development Conference Program Coordinator (15 years) (14,000 participants), and has been awarded Texas Education Agency grants ($911,000).

The post Promoting STEM to Middle School Girls through Coding and Fashion appeared first on ASEE Computers in Education Journal.

Innovative VR-Based Research to Develop Intuitive Human Computer Interaction

Francis Igot — Thu, 02 Jan 2020 21:48:20 +0000

Magesh, Chandramouli Computer

Graphics Technology
Purdue University Northwest,
Hammond, USA
magesh@pnw.edu

Abstract— This paper explains the design of a prototype desktop and augmented Virtual Reality (VR) framework as a medium to deliver instructional materials to the students in an introductory computer animation course. To be of use to other instructors or researchers interested in implementing a similar framework like this, the paper provides information on the hardware, software, and the concept inventory components of this framework. This framework was developed as part of a Teaching Innovation Grant at a Midwestern University to propose some cost-effective and innovative instructional frameworks to engage and stimulate students. This paper is an extended version of the paper presented at the CoED division of the ASEE conference and it presents VR modules and assessments with some modified techniques to the earlier version presented at the annual conference. This paper also shows the relevance of the methods used in the context of other STEM curriculum in addition to graphics and modeling courses. PC-based Desktop VR has been chosen as a medium for this study due to the ease-of-access and affordability; this framework can be visualized and accessed with the available computers in PC labs available on university campuses. Besides modeling and animation, various STEM concepts including manufacturing education, nanotechnology, chemistry, and biology are presented in an interactive manner on a desktop display. This framework allows the users to interact with the objects on the display not only via the standard mouse and keyboard, but also using multiple forms of Human Computer Interaction (HCI) such as Touchscreen, Touchpad, and 3D Mouse. Hence, the modules were developed from scratch for access via regular desktop PCs. As the author has been teaching this course for many years now, this framework has been designed with due consideration to structure of the course, ‘Introduction to Animation’. Finally, the framework has been tested on a range of VR media to check its accessibility. On the whole, this proposed framework can be used to not only teach basic modeling and animation concepts such as spatial coordinates, coordinate systems, transformation, and parametric curves, but also to teach basic graphics programming concepts.

Keywords—Virtual Reality, Interactive Instruction; STEM Education; Computer Graphics

I. INTRODUCTION AND BACKGROUND

In today’s higher educational institutions, classrooms are often overwhelmed by challenges including lack of enthusiasm and lessening attention span, which may be caused by classroom practices that fail to remedy the cognitive overload, specifically when teaching new information. Interactive VR-based tools can be quite effective in motivating curiosity and reducing cognitive load [1] [2]. Designing the instructional elements in a fun-based interactive VR setting can motivate instructors as well as students to explore new technologies and facilitate overcoming the inherent resistance to change. Over the years, VR tools and methods have been successfully employed in various educational disciplines [2] [3] [4].

Captivating students’ attention necessitates connecting to the participant (student) in the learning process and VR is a proven tool that can engage learners effectively. [3] [4] [5] [6]. According to Sherman and Craig [7], VR can be described as “… a medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment the feedback to one or more senses, giving the feeling of being mentally immersed or present in the simulation.” Several notable studies have successfully demonstrated the use VR in learning and their effectiveness in instruction [1] [3] [4] [5]. The features of VR such as interaction and navigation (Figure 1) facilitate actively engaging with the learning materials [4] [5]. The multiple modes in which contents can be delivered via VR include immersive VR, augmented VR, mixed VR, and desktop VR allow for the ability to select the correct mode as required by budgetary, infrastructure, and space constraints [3] [5]. These features are especially important in case of computer graphics (CG) materials because of the creative components involving storyboarding and production pipeline.

Figure 1. VR Characteristics & Advantages to STEM

Based on the observations in classrooms in multiple computer graphics technology (CGT) classes with students from a variety of disciplines, especially in the initial stages, students tend to be excited to create and see 3D digital objects. Often times, it is challenging for students to patiently wait to learn the underlying basics before seeing the end result. Especially, with the availability of software applications that can swiftly create 3D objects and cool software gizmos, it is hard for young minds to resist the temptation to explore a lot at once. However, succeeding in the CGT program as well as in their careers, requires a firm grasp of the fundamentals of the production pipeline and the processes therein. To overcome this challenge, it is important to balance courses with activities to allow latitude for their creativity whilst boosting critical-thinking and problem-solving skills. The lab assignments typically include a part where they would summarize their understanding of the theory and includes a creative part where they select a theme of their own choice to demonstrate their graphics skillsets. A strong understanding of the fundamental mathematics, geometric, trigonometric, and physics fundamentals plays a crucial role in determining the career-success of computer graphics (CG) students. Figure 2 shows virtual reality 3D worlds developed to demonstrates a range of concepts such as digital lighting, mechanical components, interior design, and robotics. These 3D representations (Figure 2) facilitate understanding the theoretical knowledge behind the corresponding disciplines. This program puts forth a novel PBL-based approach wherein an interactive portable desktop Virtual (dVR) framework is used to methodically organize and present foundational information. The framework uses a sequential learning process, beginning with simple concepts, gradually advancing towards more complex ideas. The medium used is desktop VR and the computers in the computer labs can be used ‘as is’. An Open Source plug-in can be used on the standard browsers such as Internet Explorer and Mozilla Firefox to view and interact with the virtual worlds.

II. VR AS AN INSTRUCTIONAL TRAINING TOOL IN STEM & NON-STEM CONTEXTS

Virtual reality (VR) and computer graphics are highly interrelated. The evolution of VR has been aided by the advancements in 3D graphics, visualization, and interactive user interfaces. Evidently, CG has tools and techniques that tremendously influence and impact the capabilities of VR and define the limitations as well. In this study, multiple modes are used to overcome space and cost limitations. Laboratories in various engineering and technology disciplines are not always able to meet the demands of today’s advanced curriculum in their conventional form with the traditional experiments. Revamping physical laboratories with equipment to specifically tailor them for pedagogical training in specialized areas is not only labor-intensive, but also costly. Virtual training laboratories, can be designed and virtual modules can be implemented across multiple platforms so that students can easily access them over standard desktop PC. Interaction with these VR worlds can be performed with standard I/O (Input/Output) devices such as a mouse and a keyboard. The interactions can be programmed to performed with other affordable input devices including a stylus^©, a touchpad^©, etc. to understand processes, assemble components, and perform trial-and-error procedures without risking equipment damage.

Figure 2. (Clockwise from Top-Left) Interactive Applications:

Lighting, Trigonometry, Interior Design, Robotics The virtual laboratories and training modules, once built, can be replicated for use on multiple machines without any additional costs. This involves copying the required software and training modules to other computers. During the training for manufacturing processes, VR tools serve as a viable alternative offering a cost and material-efficient solution by replacing the need for actual physical materials. Although virtual laboratories may not entirely replace physical laboratories, VR practices can lead to considerable cost and time savings by limiting the use of physical lab/equipment to only when actual implementation or manufacturing is required. De Jong, Linn, & Zacharia [8] state: “Virtual experiments delivered with computer technology add value to physical experiments by allowing students to explore unobservable phenomena; … to conduct multiple experiments in a short amount of time;” Virtual training has immense potential in terms of courses using alternative delivery modes (distance education) with guided voice-over and visual cues to allow students to complete tasks in a paced manner.

III. METHODOLOGY: CONCEPTS, HARDWARE & SOFTWARE

Computer animation skills are extremely important for CGT students to excel in their academic pursuit and to succeed in their career. Computer graphics are used in numerous applications including web design, print media, logo design, game design and development, 3D modeling and animation, etc. While this offers numerous opportunities for students, it inherently involves the challenge of preparing students for these opportunities. Students, especially those at the beginner level, encounter difficulties when learning to understand basic computer animation concepts and the lack of efficient tools to overcome such difficulties can affect students’ motivation. The methods described are for augmented and desktop VR. Active learning and project-based learning are proven tools to facilitate learning and retention [9] [10] as they can motivate students and facilitate applying the knowledge and skills gained in the classroom in the real-world. Instructional practices used in the course CGT 241 (Introduction to Animation) enabled the students to apply the 3D design knowledge to solve a real-world problem involving 3D printing design [11]. Pedagogical practices integrating virtual environments allow the learner to interact with the 3D representations (models). The modules are designed for access via multiple modes such as desktop VR (dVR) and augmented VR (aVR). VR training simulations have been used in a variety of fields for training outside of engineering [4] [6].

In developing VR modules, the first step involves creating a concept inventory of aspects to be covered using the 3D scenes. For example, with respect to computer graphics, a concept inventory study was carried out to determine the core concepts that would be covered using the functionalities of the VR worlds. The concepts for graphics identified included the following:

What are the references for building 3D objects- origin, coordinates, dimensions, & coordinate systems?
What are polar and Cartesian coordinate systems?
How to build shapes using points, lines, and polygons?
What are translation, rotation, and scaling?
How is uniform scaling distinct from non-uniform scaling?
What is the role of triangles & quadrilaterals in modeling?
What is poly-count and what role does it play in modeling?
How to represent in absolute and relative values?

The above represent the concept inventory questions generated for the primary stage of computer graphics learning. Similar set of questions are generated for intermediate and advanced stages of computer graphics instruction as well. The explanations to the above concepts instead of merely being provided as a book based or conventional lab-based approach are presented in the form of interactive VR worlds.

The hardware used in this research (Figure 3) consists of the following major components:

HP Z800 machine,
Oculus
Graphics Drawing Tablet
Xbox One Kinect

Figure 3. Hardware Devices Used for the Framework

The HP Z800 is a computer workstation well-known for high fidelity graphics and computations performance. The Samsung display is capable of generating binocular vision using its own proprietary glasses. The Razer Hydra is built specifically for the game industry. MS Kinect is a gaming interface designed to function with Microsoft Xbox. However, owing to the possibility of Kinect being extended (without Xbox), this research uses Kinect as the tracking interface. More specifically, we use Kinect for head-tracking of the users. It is not always required to use immersive systems like CAVE systems as affordable augmented VR (aVR). With a well-designed human computer interaction (HCI) and UI (User Interface) design can accomplish the immersion and navigation required for a specific application. It needs to be noted that not all applications that use CAVE necessarily require such high specifications. Frequently, the ultimate objective of an application can be attained using the system proposed in this research without resorting to CAVE. The display system can also use a HMD display instead of a HDTV. Head mounted displays have built features for audio, navigation controls, and tracking. Figure 4 demonstrates the essential elements of the framework developed for this study.

Figure 4. Three-Tiered Framework

The proposed framework is used as follows:

The weekly theory lessons, instead of being presented primarily in the textual format or as PDF or Power Point files, are presented via the interactive browser interface (Figure 2 – Image on top right – Interaction with trigonometry module).
The modules introduce the concepts in an interactive manner to the students. Students can practice using the interactive modules delivered in the form of 3D virtual. Then they will be led to another screen that presents the students with questions (in the form of visual demonstrations) to help them review the learning materials.
Next, the students will be led to the subsequent chapter. The modules are divided into conceptual categories such as mathematical concepts, physics concepts, trigonometric concepts, and graphics programming concepts (that help them to access as a reference if needed. The VR-based framework designed from a computer graphics perspective include the following:

A VR laboratory capable of delivering conceptual (theoretical) and practical CG training
Extensible VR modules designed to support immersion, navigation, and interaction
Coursework materials and laboratory exercises delivered in a paced manner to support face-to-face and distance- learning curriculum

The VR modules developed were also tested on multiple modes (immersive, desktop, and augmented VR). The steps involved in the process of designing and implementing the framework have been summarized into three major procedures (Figure 5):

The 3D models are created using a software (Autodesk 3ds Max or Maya) or programming language such as Java3DL
Export to a compatible format such as .OBJ (Object) or. FBX (Film box) to be compatible with Unity
Program and customize objects in Unity

Figure 5. Major Steps in Tier II Model Creation

Figure 6 below demonstrates the interactive delivery of a module to explain programming concepts [1]. There are nine icons showing various elements of the learning module. The icon on the bottom right (x=3) is used to demonstrate the concept of data types in programming.

Figure 6. Step-by-Step Interaction Using VR

The laptop (Figure 6) shows the actual module being executed. The diagram on the right shows the step-by-step interactive game where the data items such as integer, float, char, and string are matched using drag and drop into the correct cylinders. As the student selects the correct data types and dropping them into the correct cylinders, they start moving down and finally when all the items are correctly matched the cylinders are finally all the way down showing the module has been completed.

IV. RESULTS AND DISCUSSION

This section discusses the hardware and the software issues in addition to the actual results from the 3D interactive VR displays. Selecting the correct VR system is a multifaceted problem. The system must be able to meet the instructional, graphics-based, immersive, and interactive aspects necessary for users to receive necessary instruction while being immersed in the simulation. This is to balance educational necessity with the goal of motivating learners with interaction and graphics [12]. Factors considered include:

Hardware (CPU) and graphics requirements: System requirements were considered to decide if available computers are compatible with the system or if they will require a better graphics cards, CPU, etc. As the visual learning style is critical, the system requires the necessary tools for learners to interact with the simulation [1].
Cost: Economical systems are vital to accomplish the intended goal, as staying within budget is necessary.
Standalone Capability: can function free of costly additional hardware/devices without server support.
Software compatibility/Support: Software compatibility/support refers to the support from the company/community that the system is associated with.

Unity was used as the development platform due to the support of this platform from companies and online communities, and is recognized as a common development language. Unity works very well with VR due to the Unity VR package and compatibility with HMD and desktop VR. The options that were considered for the VR system were the Samsung Odyssey, Google Card Board Headset (GCBH), and Dell Visor (Figure 7). In order to determine the best system for the study, a comparative analysis was created using the Oculus/HTC as the basis to compare the other systems.

Figure 7. VR systems used for Testing VR modules

Table 1 shows the summary of the comparative analysis of the VR systems while comparing characteristics including cost, standalone capability and software requirements. Using a virtual workflow, basic 3D operations were explained through interacting with the objects in the 3D environment.

Table 1. Comparative Analysis of VR Devices

	COST	STANDALONE	SOFTWARE
Oculus/ HTC Vive	$500 Vive $400 Oculus Laptop Needed	Need additional laptop	High graphic capability, Interaction
Samsung Odyssey	$400 system Phone $300-500 plan	Smartphone with plan Needed	Accurate controls, Needs smart phone
Google Cardboard	$15 Cardboard, Phone $300-500, plan varies	Requires a modern smartphone with a plan	Limited interaction, most affordable
Dell Visor	$300 Visor, $15 dongle, $10 adaptor	Standalone (low- cost Adaptor and dongle)	Accurate programmable controls.

Figure 8. Virtual Workflow to facilitate Interactive Learning of 3D Transformations

To explain transformational concepts, a simple and practical example for chosen and students interact with this in the VR scene. Students in a university are typically well- acquainted with the library setup and are familiar with the library environment. Hence, a 3D VR-based common example of stacking books and arranging the layout is used to explain the fundamental concepts of transformation (Figure 8). A complete library of 3D objects has been built to help students help students understand the basic concepts demonstrated earlier has been developed so that students can use such real-world examples to understand and practice the above theoretical aspects (Figure 9).

Figure 9. Virtual Library –Showing a part/section

Tables 2 and 3 demonstrate the beginner and intermediate/advanced VR scenes generated to explain the computer graphics concepts to the students. These virtual worlds are capable of interaction using the standard devices and those mentioned above Wacom © and Dell © Touchpad. These are relatively much cheaper and can be procured and installed without much difficulty. Other additional features of interaction were using affordable devices including the following:

Standard Gaming Mouse/Keyboard
Wacom © Tablet/Stylus
Dell © Touchpad
Razer Hydra ©

Table 2 provides an overview some of the CG concepts. Table 3 provides an overview of some of the intermediate to advanced CG concepts that have been covered within the VR framework that can be accessed via multiple modes. This system can be used to facilitate instruction of STEM concepts through interactive learning exercises and active learning in Engineering and Technology curriculum. In addition, students in various ET disciplines can use this framework to apply CG concepts in their discipline-specific applications. Various examples were discussed earlier demonstrating components that were assembled together to form a scene assembly thereby enabling students to understand the concept of transformation. The salient features of the framework and how these are aligned with the course learning outcomes is depicted in Table 1.

In summary, the salient features and the advantages of the framework are as follows:

Simple approach aiding coherent flow
VR examples to explain CG concepts
Active learning via interaction to facilitate underlying mathematical concepts
An easy to understand, VR- based approach for explaining advanced concepts

V. CONCLUSION

This framework allows the users to interact with the objects on the VR display not only via the standard mouse and keyboard, but also using multiple forms of HCI such as Touchscreen, Touchpad, and 3D Mouse. Hence, the modules were developed from scratch for access via regular desktop PCs. As the author has been teaching this course for many years, this framework has been designed with consideration to structure of the course, ‘Introduction to Animation’. This project employs a pedagogical approach using multi-modal VR for CG instruction specifically targeted at 4-year degree programs. In addition to CG (computer graphics), 3D modeling and animation courses are also taught in various other engineering and technology disciplines. In this paper, the framework discussed is primarily from the point of view of a technology curriculum with more emphasis on the application. However, some basic theoretical aspects of modeling and animation are inevitable in order to create 3D models and animate them. This work represents an extended version of the paper presented at the CoED division in the ASEE conference. This framework has improved in the following ways:

More streamlined approach for the design and implementation of the VR modules
Wide range of examples in different STEM disciplines
Enhancing the framework with Unity platform interaction and programming Customization for specific Head Mounted Displays

ACKNOWLEDGEMENTS

The author expresses gratitude to the PNW Teaching Innovation Grant and the PNW Internal Grants that facilitated this research. This effort was also partly funded by the NSF Grant # 1700674.

VI. LIMITATIONS

There were some important limitations and/or constraints that the authors experienced with this prototype study. The vast majority of the efforts in the study were dedicated towards designing the framework, development, and implementation with consideration of specific CG curriculum.

Studies were conducted to develop and test applications in diverse disciplines such as other branches of ET, Interior Design, Manufacturing, Civil Engineering etc. However, as part of future studies, the authors endeavor to conduct tests and include data related to student performance. These studies would specifically target measuring the accomplishment of course learning outcomes within the ET disciplines.

As this is a work in progress, the framework needs to be further developed and then these modules will be tested in CGT courses with Institutional Review Board (IRB) approval. Also, the basic concepts in computer graphics span over a wide range of topics that vary according to the software languages and modeling platforms. Based on the literature review this study has narrowed down the very basic concepts that are common to different CG courses across the nation. Detailed feedback will be obtained from the participants and comprehensive quantitative/qualitative evaluations will be carried out. Furthermore, focus groups of participants will be included during the workshops to actively engage other instructors as well and solicit their critiques on methods implemented thus far and for future enhancement.

REFERENCES

[1] Chandramouli, M., & Heffron, J. (2015, March). A desktop vr-based HCI framework for programming instruction. In 2015 IEEE Integrated STEM Education Conference (pp. 129-134). IEEE.

[2] Dias, P., Sousa, T., Parracho, J., Cardoso, I., Monteiro, A., & Santos, B. S. (2014). Student projects involving novel interaction with large displays. IEEE computer graphics and applications, 34(2), 80-86.

[3] Toth, E. E., Ludvico, L. R., & Morrow, B. L. (2014). Blended inquiry with hands-on and virtual laboratories: the role of perceptual features during knowledge construction. Interactive Learning Environments, 22(5), 614-630.

[4] Chandramouli, M., Zahraee, M., & Winer, C. (2014, June). A fun-learning approach to programming: An adaptive Virtual Reality (VR) platform to teach programming to engineering students. In IEEE International Conference on Electro/Information Technology. IEEE

[5] Jen, Y. H., Taha, Z., & Vui, L. J. (2008). VR-Based robot programming simulation system for an industrial robot. International Jl. of Industrial Engineering, 15(3), 314-322.

[6] Pan, Z., Cheok, A. D., Yang, H., Zhu, J., & Shi, J. (2006). Virtual reality and mixed reality for virtual learning environments. Computers & graphics, 30(1), 20-28.

[7] Sherman, W. R., & Craig, A. B. (2003). Understanding virtual reality. San Francisco, CA: Morgan Kauffman.

[8] De Jong, T., Linn, M. C., & Zacharia, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305-308.

[9] Jou, M., & Wang, J. (2013). Investigation of effects of virtual reality environments on learning performance of technical skills. Computers in Human Behavior, 29(2), 433- 438.

[10] Monahan, T., McArdle, G., & Bertolotto, M. (2008). Virtual reality for collaborative e-learning. Computers & Education, 50(4), 1339-1353.

[11] Houstex International 3D Design Competition houstexonline.com/3d-printing-student-competition-winners/

[12] Chandramouli, M., Takahashi, G., & Bertoline, G. R. (2014). Desktop VR centered project-based learning in ET courses using a low-cost portable VR system. In Proceedings of the American Society of Engineering Education.

The post Innovative VR-Based Research to Develop Intuitive Human Computer Interaction appeared first on ASEE Computers in Education Journal.

A Pattern Recognition Framework for Embedded Systems

Francis Igot — Fri, 03 Jan 2020 22:10:41 +0000

Frank Vahid

Tony Givargis

Roman Lysecky

Abstract— Embedded systems often implement behavior for common application domains, such as the control systems domain or the signal processing domain. An increasingly common domain is pattern recognition, such as determining which kind of fruit is passing on a conveyor belt. Embedded system students and designers typically are not experts in such domains and could benefit from simpler platforms to help them gain insight into the problem of pattern recognition and help them develop such algorithms rapidly. Generic frameworks, such as PID (proportional- integral-derivative) for control, or FIR (finite impulse response) for signal filtering, empower non-expert embedded system designers to quickly build robust systems in those domains. We introduce a generic pattern recognition framework, useful for education as well as for various real systems. The framework divides the task into three phases: feature extraction, classification, and actuation (FCA). We provide template code (in C) that a student or designer can modify for their own specific application. We show that the FCA pattern recognition framework can readily be adapted for various pattern recognition applications, like recognizing box sizes, fruit type, mug type, or detecting vending machine vandalism, requiring only 2-3 hours to create each new application. We report results of a randomized controlled study with 66 students in an intermediate embedded systems class, showing that the framework could be learned in tens of minutes and yielding applications with higher recognition accuracy of 71% for pattern recognition vs. 57% without the framework (p-value=0.03).

Index Terms—Embedded Systems, Pattern Recognition, Teaching framework, Computer Science Education

I. INTRODUCTION

While many embedded system applications are unique, others implement behavior of well-known domains, illustrated in Figure 1.

For example, heating an oven to a target temperature, or propelling a small robotic car at a target speed, both represent instances of control systems. A control system [1] strives to control a physical system to match an actual physical feature (temperature, speed) to a target value. A rich discipline of control theory exists, with various mathematical techniques for modeling and controlling physical systems aiming to best match the target (minimizing overshoot, oscillations, and steady-state error) and ensuring stability. However, such theory requires extensive knowledge and training, which many embedded systems students lack early in their careers. Thus, a generic framework for control has been developed, known as PID (proportional-integral- derivative) control [2] to empower non-experts and students to build good quality control systems. The simple concept of PID, plus template code, and techniques for tuning P, I, and D variables in such code, quickly yield systems of sufficient quality for various applications. Without domain experts or knowledge, embedded system students and designers might otherwise develop low-quality control solutions; PID enables such designers to build quality solutions, in the same or even less time as they would otherwise.

Similarly, removing noise from a digital signal, or letting only a certain frequency band pass, both represent instances of digital signal filtering. An FIR (finite impulse response) filter [3] is an easy-to-understand generic framework for filtering in software, allowing non-experts and students to modify variables in template code to implement specific filtering applications.

PID and FIR frameworks each build on existing domain theory to provide simple but powerful methods for embedded system designers and students. Meanwhile, embedded systems are increasingly used to recognize specific patterns. In general, a pattern recognition system takes as input data for an object, and outputs a category for that object. For example, a common such system takes a face photograph as input, and outputs a known person’s name. Most work on pattern recognition systems have a desktop computing model, where input and output are via a file or database. In contrast, a pattern recognizing embedded system, illustrated in Figure 2, gets input from sensors, like weight or color sensors, and generates output by controlling actuators, like changing a directional gate on a conveyor belt, all in real- time. For example, a warehouse may use a pattern recognition system to recognize whether a box on a conveyor belt is one of three sizes, keeping count for inventory purposes. A grocery system may recognize the kind of fruit on a scale (apple, pear, banana) as in Figure 2 and output a total price based on kind and weight.

A front-door camera system may detect whether a person is a known person from a database and generate a different doorbell ring for known persons. A wrist-worn device may recognize that an elderly person has fallen and alert caregivers [4]. An embedded camera may be used to detect a red peach [5] on a tree for harvesting. A package express center may use an embedded magnetic matrix and outside sensors to monitor conveyor belt parameters such as tension, load, and position of objects [6]. A system may detect the surface defects on fruits to separate bad fruit from good fruit [7]. A system may be used for detecting the size and grading a fruit to determine its quality [8] for pricing purposes. Governmental organizations may predict floods using pressure and ultrasound flow sensors attached to embedded computer nodes to reduce damages caused by flooding [9]. A traffic system may choose the proper light at intersections to tackle the traffic congestion problem using controllers in traffic lights and sensors on the road [10]. As can be seen, pattern recognition is becoming another common application domain in embedded systems.

Like control systems and signal filtering, pattern recognition is an established domain with extensive theory and techniques. However, many (if not most) embedded systems students and designers do not have that domain knowledge. The pattern recognition domain has dozens of techniques that can overwhelm a student or an embedded system designer who tries to quickly learn about the domain to improve their embedded application. Without access to domain experts, designers may build low-quality pattern recognition systems.

This paper introduces the FCA (feature extraction, classification, actuation) framework for pattern recognition in embedded systems, describes the template code, and summarizes results of using the FCA framework in a classroom setting.

II. FCA FRAMEWORK

Years of engaging with commercial embedded systems applications through various consulting and other arrangements has led to our observation that many embedded applications handle pattern recognition poorly. Some other researchers similarly state the issue [26], with some pointing to issues like limited resources making the pattern recognition problem harder [24], and to the lack of good embedded systems curriculum [25]. In some cases, the developers are not aware of the pattern recognition field or do not realize that their application is performing pattern recognition, and thus the developers do not attempt to draw upon established pattern recognition techniques. Instead, their software may simply use a series of conditions implemented via if-else statements, or a finite state machine, to categorize data coming from sensors. Or, developers may choose a pattern recognition technique but use it poorly. Furthermore, developers may attempt such categorization on raw sensor data. Such software may be making decisions based on too-detailed highly varying input data and may also have actuation distributed throughout the code. The result is a complex piece of software that is hard to update (such as if new sensors are introduced) and that may have low accuracy.

Thus, our first step was to define a modular process specifically for pattern recognition in embedded systems. The process divides pattern recognition into three phases —feature extraction, classification, and actuation, or FCA —shown in Figure 3, whose parts are summarized below.

Sensors: The input consists of a combination of various sensors, differing depending on the application. One application may have weight and color sensors. Another application may have color and infrared (IR) break beam sensors. Another may have accelerometers. A designer determines which sensors to use to help with pattern recognition tasks.

Feature extraction: The first phase in pattern recognition is to convert raw sensor data into features that will guide classification. For example, 3-dimensional accelerometer data may be converted into a velocity vector or an acceleration magnitude. Data from several IR sensors may be combined to estimate length, width, and height features, or converted into a volume feature. This phase prevents the common situation of embedded systems designers trying to use raw sensor data directly to make classification decisions. Feature extraction simplifies classification by pre- shaping the data to represent information most useful to classification. For example, in package sorting, box height is more directly useful than data on how many of four vertical IR sensors currently have broken beams (assume multiple IR sensors are stacked at increasing height and a moving box will break a number of the beams depending on its height). The output of feature extraction consists of values for a set of features, such as “height = 3.5 in” and “red = 200” (indicating the amount of red on a scale of 0-255).

Classification: Given values for a set of features of an object, classification determines to which category the object most likely belongs. The output is a single category, such as “apple”, or “small box”.

Actuation: Given a category, actuation takes an appropriate action by setting values of an actuator. If a “small box” is detected in a warehouse system, a stepper motor (a kind of actuator) may be set to guide the box on a conveyor belt to a particular bin. If “apple” is detected in a grocery system, a variable may be set to an appropriate price based on the fruit’s weight.

While seemingly simple, this division into three phases enforces a modularity that developers otherwise might bypass. The division disallows using any raw sensor data in classification, requiring instead that such data be pre-shaped into well-defined features. The division also disallows setting actuators throughout the code; instead, all actuator setting is done after classification is completed. An analog might be made with programming, wherein a function should only read/write its parameters and not global variables – a concept that was not well-understood in the early days of programming, whereas today the importance of modularity with respect to functions is well understood. Using raw sensor data or setting an actuator in the classification code is analogous to reading or writing a global variable in a function, both of which can have unintended side effects and may result in complex hard-to- maintain code. Establishing such modularity is one goal of the FCA framework.

III. CLASSIFICATION

Pattern recognition is a widely studied domain, stemming from the fields of computer science and electrical engineering. The domain goes by various names, including pattern recognition, machine learning, data mining, and knowledge discovery, with papers dating back to the 1960s. As a result, a multitude of techniques exist.

The techniques of most interest to embedded system designers are those known as supervised learning [11], in particular classification. In classification, a set of training objects is provided, with each object labeled as being in a certain category. Then, given a new object, a classification technique, based on the training set, strives to determine in which category the new object most likely belongs, such as: apple, pear, or banana.

For an embedded system designer, trying to determine which classification technique to apply can be overwhelming. For example, the Wikipedia [12][23] entry on supervised learning states that no technique is best and lists many, each technique having extensive literature:

Support vector machines
Linear regression
Logistic regression
Naive Bayes
Linear discriminant analysis
Decision trees
K-nearest neighbors
Neural networks

An embedded system designer can be quickly overwhelmed by all the options. Thus, we strove to determine if one technique could serve as a good basis for a generic framework in embedded systems. Our criteria included:

Simple to learn by embedded system designers
Good classification accuracy across various common embedded system applications
Efficiently implemented on resource- constrained embedded platforms

Below, we review K-nearest neighbors, logistic regression, naive Bayes, decision trees, support vector machines, and neural networks along with their advantages and disadvantages based on our criteria.

K-nearest neighbor (KNN) [13] is a simple, easily understandable classification algorithm. KNN places known objects in an N- dimensional space, where each dimension is a feature (weight, height, color, etc.). Then, to classify a new object, KNN determines the K closest neighbors in the space; the most common category among those K neighbors wins. Figure 4 illustrates. KNN is powerful and can handle non-linearity and multiclass classification. Moreover, by having a large enough training set, the error rate of KNN would be less than twice the minimum achievable error rate [14]. But, in case of having a large training set, this algorithm can be inefficient in terms of processing time/memory due to the need for storing and searching the training set.

Logistic regression [15] is a linear binary classification algorithm that statistically predicts the odds of an object being in a class based on the features. Logistic regression is fast and efficient in production. However, in this algorithm, objects of different classes should be linearly separable. Therefore, to make the algorithm work well in various embedded systems applications, an additional complex feature representation step is required before classification.

Naive Bayes [16] is a classification algorithm designed based on Bayes’ theorem, and learns the distribution of the input data to predict the probability that an input object belongs to a particular class. Naive Bayes is simple both in concept and implementation and is efficient in production. However, it makes a strong assumption of independence between features of the input data and it suffers in performance when data is sparse in some classes or features, which makes it incompatible with many real- world embedded system applications.

Decision trees [17] are a nonlinear classification technique, which learns simple decision rules from the training set to classify a new input object. Decision trees are easy to interpret and understand. However, decision trees are prone to overfitting (matching particular data too closely, thus being not general) and lack robustness due to high variance in classification accuracy. To increase robustness, one can use an ensemble of decision trees, but that makes the technique more complex for embedded system development.

Support vector machines [18] are a binary classification algorithm, which classifies objects by finding the hyperplane that represents the largest separation between two classes and maximizing their margin. The technique is robust and demonstrated good quality in various embedded systems applications [19, 20, 21]. Moreover, the technique is efficient in production in terms of memory and processing speed. However, the technique’s performance heavily relies on a kernel function, which is not easy to choose especially for non-expert embedded systems developers. Given that support vector machines are mainly binary classifiers and their extension to ternary and higher classification is not very effective, they have limited applicability in the embedded applications.

Neural networks [22] are inspired by the human brain and nervous system. The technique is powerful and has achieved much of the state of the art in the image processing and natural language processing fields. However, the technique is complicated to understand and implement. Moreover, training requires a large amount of data, which often is not available in embedded systems applications.

After some investigation, discussions, and many recommendations from various classification researchers, we chose KNN as the most appropriate general classification technique for embedded systems developers. KNN is highly intuitive: most people can understand the algorithm just by looking at Figure 4. The algorithm is straightforward to code and has small code size and is efficient for moderately-sized training sets as is common in embedded systems. For larger training sets, techniques exist to improve efficiency, such as aggregating objects, caching previous results, etc. The algorithm has demonstrated high accuracy. These features make it a popular choice in general pattern recognition as well but are especially useful in embedded systems due to limitations on code resources (speed, size) and due to ease of understanding and modification by embedded developers.

IV. TEMPLATE CODE

A simple but powerful software productivity improvement techniques is to provide working template code (called “reference code” in many domains) that a developer can modify for his/her particular application. We thus developed template code in C to support the FCA framework. A designer can adjust the code to carry out their own pattern recognition application. Highlights of the code are shown below (the appendix has more complete code); various helper functions and other items have been omitted. A key aspect to notice is the code’s simple organization, enabling a non- expert embedded developer to readily understand, navigate, and modify the code.

The template code shown in Figure 5 begins by defining a categories array. It then defines a structure for an object (either in the training set, or the new object to be classified), with fields being the object’s category and the various features of the object. It next defines a training set, with functions for populating the set.

Figure 5. Template Code

Next, the FeatureExtraction function samples input sensor values and converts those values into desired features. The FeatureExtraction function performs a simple unit conversion. Often, the conversion may involve combining values from multiple sensors, such as converting three x, y, and z accelerometer sensor values into the two features of acceleration direction and acceleration magnitude. FeatureExtraction ends by calling the RescaleObject function to scale all features to values between 0 and 1, so that “distances” make sense in the subsequent KNN algorithm. The RescaleObject function is omitted.

Next, the ComputeDistanceOfObjects function computes the Euclidean distance of any two objects, based on their N (scaled) features in an N-dimensional space. That function is used by the subsequent Classification function, which computes the distance between a new object and every object in the training set, to determine the K nearest training objects, and then returning the most frequent category for those K objects.

Next, the Actuation function sets outputs (actuators) in response to a given category.

Finally, the main function populates the training set, sets up a sampling rate, and then repeatedly calls FeatureExtraction, Classification, and Actuation at that rate (using a microcontroller’s built-in timer or whatever timing services are provided).

V. ADAPTING THE TEMPLATE FOR VARIOUS APPLICATIONS

We carried out various adaptations of the applications, to gauge the effort required and effectiveness of the resulting system.

The first was an application from a factory warehouse. The system was required to detect which of three sized boxes were on a conveyor belt and direct each box to a different bin for each size. We initially considered a video-based system but decided a simpler approach would be preferable. We used a break beam sensor to detect a box’s presence, and five distance sensors: one to measure height, one to measure width, and two to measure length. The feature extractor included a simple state machine: When the beam was broken, a new box was arriving. When that beam was no longer broken, we considered all the distance sensor values before that moment. With those values, we computed height, width, and length features of the box. For training, we ran the three-sized boxes past the sensors in various orientations and recorded the height, width and length values, yielding 9 training data values. We entered those values into the array of known objects in the template. The entire process of modifying the template code and populating the known objects array with training data required 3 hours. We tested the system with 9 new boxes and obtained accuracy of 100% (9/9 correct classifications).

The second was a grocery store application to detect which types of fruits were passing through on a conveyor belt. The fruits were apples, oranges, lemons, and pears. We used a break beam sensor to detect a fruit’s presence, plus a weight sensor and color sensor. For training, we ran 4 of each fruit through the system to record the fruits’ weights and colors, giving us 16 training values. These values were then entered into the array of known objects in the template. This process took 2.5 hours: one hour to modify the template and the rest spent running and entering the training values. We tested the system with 20 fruits and obtained 19/20 or 95% accuracy – one pear was marked as an apple when its bruised side faced the color sensor.

The third application involved putting a mug on a scale to detect if the cup was an 11 oz mug, a 16 oz mug, or a travel mug. A break beam sensor started the system. We used three distance sensors, two to measure the width of the mug and one to measure the height, and a weight sensor. We trained by sending nine mugs through the system, three of each type. These values were entered into the array of known objects in the template. The process of modifying this code took just under two hours due to us already having the code for the distance sensors and the weight sensor. We tested the system with 6 new mugs and obtained 6/6 or 100% accuracy.

Another application was detecting whether or not a vending machine was being tampered with. In contrast to the above applications that used a break beam sensor to start the pattern recognition process, this application ran the feature extraction part every one second. We used an accelerometer as the only sensor. The sensor provided three values: change in the x axis, change in the y axis, and change in the z axis. Every second, one hundred values were measured with a delay of 10 milliseconds between each measurement. We used a formula to measure the largest change in each axis and these changes were added to the training data, ending with 8 training sets in all, categorized as “normal shaking” or “abnormal shaking”. This application took 4 hours to modify due to the feature extraction portion being different from the other applications. We then gave the device different shakes and obtained 17/20 or 85% accuracy. Here, “accuracy” is less well defined, because humans are the ones defining and judging normal and abnormal shaking, both during training and testing.

It should be noted that, ultimately, the quality of a recognizer’s robustness and accuracy relies on having robust training datasets.

VI. RANDOMIZED CONTROLLED STUDY

To further evaluate the framework’s usability, we conducted an experiment with all 66 students in the Intermediate Embedded and Real-Time Systems class at the (university name withheld for blind review). We randomly divided participants into two groups, A and B.

Both groups started at an “Instructions” web page indicating that they would be participating in an embedded systems coding experiment, and that they were required to work independently and not refer to any outside resources; teaching assistants were present to ensure those rules were met. The instructions indicated that they had 45 minutes, that they were afterwards be given a survey, and that their submissions were anonymized (i.e., no impact on their course grade). Group A had one additional instruction, asking students to complete the tutorial available on the next page, before attempting to code.

Upon clicking “Start experiment”, all students were taken to an “Experiment” page. At the top was a problem description asking to solve a problem of classifying a person to a man, woman, or child category based on the height and weight of the person using training data for 18 people. Beneath the problem statement was an embedded systems simulator where students could write C code to read from microcontroller inputs and write to outputs; students had been using this simulator throughout the quarter already. The simulator showed its standard code for both groups, consisting of required includes at the top, and a main function consisting of a timer initialization followed by an infinite while(1) loop.

The only difference between the groups here was that Group A saw a button at the top labeled “Tutorial”, while that button was not present for Group B. The button for Group A led to a brief discussion on pattern recognition along with FCA template code and an example. Note that the instructions did not tell Group A students that they were required to use the template code.

Upon reaching that experiments page, a timer at the top of the page immediately started counting up from 0 minutes. The teaching assistants ensured all students stopped before reaching 45 minutes.

We evaluated the submitted models on one hundred test cases. As we assumed (and hoped), all Group A students chose to use the FCA template code in their solutions. Table 1 shows the average accuracy and average time spent to solve the problem for each group. The comparison verifies that access to the classification resources in group A led to the development of significantly more accurate models (p=0.03 using an independent t-test), and that the provided template was quickly learnable.

Figure 6 provides a detailed view of the accuracies obtained by each group; Group A had many more applications with 80-100% accuracy, and none with 0-20% accuracy that occurred for some in Group B.

Analysis of the submission time information shows that participants in group A spent slightly higher time to solve the problem compared to group B. This is mostly due to group A needing to take extra time to read the tutorial and review and understand the reference code, which is a one-time overhead that would be reduced if the individual used the framework again.

Also, a more detailed study of submitted models shows that similar to the sample model displayed in Figure 7, all participants in group B used a series of branch statements to solve the problem (as we had observed in various commercial applications, mentioned in Section 2). In contrast, all participants in group A copy- pasted and modified the template code, as in Figure 8, even though the default code provided to group A was only a basic main() function with a timer and infinite loop, identical to group B.

Discussion: Some might say that Group A of course would do better because they received a tutorial and template code. However, such better performance was not a forgone conclusion. If FCA was not easy to understand, and if the code template was large or confusing,

Group A students might have chosen to ignore the FCA code and write their own code, as Group B did. Or, Group A students might have tried to use the FCA template code but failed to complete the task or created inferior solutions. Indeed, many of the pattern recognition techniques would have been almost impossible to teach and gain success in such a short period of time. We intended to show that FCA is quickly learnable and effective, and the results seem to support that hypothesis.

After submitting the code, participants took a survey with seven questions regarding solving the problem and the tool. Each question in the survey had six options representing the level of agreement with the question statement. We converted those options into numbers from 1 to 6, where 1 corresponds to the strongly disagree option, and 6 corresponds to the strongly agree option. Table 2 shows the weighted average responses of each group. The results show a slight improvement across 4 of the 5 questions, though not statistically significant except for the last question. The conclusion here is that the extra work of the FCA tutorial in the short available time did not create any extra stress or confusion.

VII. RANDOMIZED CONTROLLED STUDY

Pattern recognition applications continue to grow in embedded systems. We developed a framework to enable embedded systems students and designers to readily build robust pattern recognition applications, without having pattern recognition domain expertise. The framework divides pattern recognition into three phases: feature extraction, classification, and actuation (FCA). We chose K-nearest neighbors for classification due to simplicity and robustness. We developed template code in C for the FCA framework with great emphasis on simplicity.

We found that we could adapt the template code to four different applications in just a few hours for each, with highly accurate classification. We conducted an experiment showing students could readily understand the code and adapt it for a given application, yielding significantly improved recognition accuracy.

Embedded system students typically are not experts in such domains and could benefit from simpler platforms to help them gain insight into the problem of pattern recognition and help them develop such algorithms rapidly. This work contributes to this educational mission. Furthermore, many embedded systems engineers are non-experts as well. This work enables engineers to build robust pattern recognition systems without being an expert in the pattern recognition field, akin to how PID control and FIR filters enable engineers to build robust control or filtering systems without being experts in those fields.

APPENDIX

This appendix provides a nearly complete listing of the template code, as shown in Figure 9. Much attention was paid to keeping the code as short and simple as possible, to enable rapid understanding by embedded developers.

Figure 9. Nearly complete code listing.

ACKNOWLEDGEMENT

This work was supported in part by the National Science Foundation (NSF grant number 1563652). We thank Shayan Salehian and Bailey Herms, whose masters projects contributed to this work.

REFERENCES

[1] Dorf, R. C. and Bishop, R. H. (2011). Modern control systems. Pearson.

[2] Wescott, T. (2000). PID without a PhD. Embedded Systems Programming, 13(11), 1-7.

[3] Wagner, B. and Barr, M. (2002). Introduction to digital filters. Embedded Systems Programming, 47.

[4] Bourke, A. K., O’brien, J. V., and Lyons, G. M. (2007). Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. Gait & posture, 26(2), 194-199.

[5] Teixidó, M., Font, D., Pallejà, T., Tresanchez, M., Nogués, M., and Palacín, J. (2012). An embedded real-time red peach detection system based on an OV7670 camera, arm Cortex- M4 processor and 3D look-up tables. Sensors, 12(10), 14129-14143.

[6] Pang, Y. and Lodewijks, G. (2006, June). A novel embedded conductive detection system for intelligent conveyor belt monitoring. In International Conference on Service Operations and Logistics, and Informatics, pp. 803- 808). IEEE.

[7] Davenel, A., Guizard, C. H., Labarre, T., and Sevila, F. (1988). Automatic detection of surface defects on fruit by using a vision system. Journal of Agricultural Engineering Research, 41(1), 1-9.

[8] Dang, H., Song, J., and Guo, Q. (2010, August). A fruit size detecting and grading system based on image processing. In Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2010 2nd International Conference on (Vol. 2, pp. 83-86). IEEE.

[9] Hughes, D., Greenwood, P., Coulson, G., and Blair, G. (2006). Gridstix: Supporting flood prediction using embedded hardware and next generation grid middleware. In World of Wireless, Mobile and Multimedia Networks, 2006. WoWMoM 2006. International Symposium on a (pp. 6-pp). IEEE.

[10] Thakare, V. S., Jadhav, S. R., Sayyed, S. G., and Pawar, P. V. (2013). Design of smart traffic light controller using embedded system. IOSR Journal of Computer Engineering (IOSR-JCE), 10(1), 30-33.

[11] Russell, S. J., Norvig, P., Canny, J. F., Malik, J. M., & Edwards, D. D. (2003). Artificial intelligence: a modern approach (Vol. 2, No. 9). Upper Saddle River: Prentice hall.

[12] Supervised learning. (2018, February 28). Retrieved March 08, 2018, from https://en.wikipedia.org/wiki/Supervised_learning

[13] Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1), 21-27.

[14] Elkan, C. (2011, January). Nearest neighbor classification. elkan@cs.ucsd.edu.

[15] Hosmer Jr, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). Applied logistic regression (Vol. 398). John Wiley & Sons.

[16] Theodoridis, S. and Koutroumbas, K. (2008). Pattern Recognition. Elsevier Science.

[17] Duda, R. O., Hart, P. E., and Stork, D. G. (2012). Pattern classification. John Wiley & Sons.

[18] Cristianini, N. and Shawe-Taylor, J. (2000). An introduction to support vector machines and other kernel- based learning methods. Cambridge university press.

[19] Shi, G., Chan, C. S., Li, W. J., Leung, K. S., Zou, Y., and Jin, Y. (2009). Mobile human airbag system for fall protection using MEMS sensors and embedded SVM classifier. IEEE Sensors Journal, 9(5).

[20] Meng, H., Pears, N., & Bailey, C. (2007, June). A human action recognition system for embedded computer vision application. In Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on (pp. 1-6). IEEE.

[21] He, Z. Y. & Jin, L. W. (2008, July). Activity recognition from acceleration data using AR model representation and SVM. In Machine Learning and Cybernetics, 2008 International Conference on (Vol. 4, pp. 2245-2250). IEEE.

[22] Hagan, M. T., Demuth, H. B., and Beale, M. H. (1996). Neural network design (Vol. 20). Boston: Pws Pub.

[23] Wolpert, D. H., Macready, W. G., No Free Lunch Theorems for Search, Technical Report SFI-TR-95-02-010, 1995.

[24] Ring, Matthias, et al. “Software-based performance and complexity analysis for the design of embedded classification systems.” Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). IEEE, 2012.

[25] Ricks, K., D. Jackson, and W. Stapleton. Incorporating embedded programming skills into an ECE curriculum. ACM SIGBED Review, January 2007.

[26] Perez-Cortes, JC, JL Guardiola, and AJ Perez-Jimenez. Pattern Recognition with Embedded Systems Technology: A Survey. 20th Int. Workshop on Database and Expert Systems Application, 2009.

The post A Pattern Recognition Framework for Embedded Systems appeared first on ASEE Computers in Education Journal.

Project-Based Courses for B.Tech. Program of Robotics in Mechanical Engineering Technology

Francis Igot — Sat, 04 Jan 2020 22:18:04 +0000

Zhou Zhang

Department of Mechanical Engineering Technology
New York City College of Technology, CUNY
Brooklyn, New York, USA zhzhang@citytech.cuny.edu

Andy S. Zhang

Department of Mechanical Engineering Technology
New York City College of Technology, CUNY
Brooklyn, New York, USA azhang@citytech.cuny.edu

Mingshao Zhang

Department of Mechanical and Industrial Engineering
Southern Illinois University Edwardsville
Edwardsville, Illinois, USA mzhang@siue.edu

Sven Esche

Department of Mechanical Engineering
Stevens Institute of Technology Hoboken, New Jersey, USA sesche@stevens.edu

Abstract— Robotics program at many Colleges has continued to become more and more popular. However, the students of the Bachelor of Technology (B.Tech.) program of robotics in the Mechanical Engineering Technology (MET) are facing three difficulties: (1) Weak fundamental knowledge related electrical engineering (EE), computer science (CS) and information technology (IT); (2) Difficulty in understanding the advanced concepts and theories of robotics; (3) Limited robotics class hours. Therefore, devising a series of appropriate robotics classes for the MET program is desirable.

In order to overcome the above-mentioned problems, four project-based courses for the robotics program is devised and implemented in the Department of MET. There are three levels of robotics courses ranging from ‘introduction’, ‘application’ to ‘advanced’. A series of projects corresponding to different levels are designed and then are assigned to students. The students learn and practice the fundamental theories of robotics through projects instead of mathematical analysis. These courses have two advantages. First, the projects let the students understand the theories spontaneously and expand the given projects with these theories. Second, the goal of the proposed courses is to release the dependency on advanced algorithms and optimization. Then, the students can familiarize themselves with the principal concepts of robotics, practice the application of hardware and software, create their own innovative projects and, prepare themselves for their entries into the job market, thus supporting the central educational goal of cultivating technologists in MET.

Keywords— Robotics, Mechanical Engineering Technology, Curricula Design

I. INTRODUCTION

The fields of engineering and engineering technology have been broadened significantly by a number of emerging topics including additive manufacturing[1], internet of things (IoT)[2], artificial intelligence (AI)[3], virtual reality (VR)[4], etc. Currently, robotics is now integrating so many cutting-edge topics together to contribute to the world-wide innovations. Moreover, robotics has brought about a revolution at the Colleges. On one hand, the graduates of robotics gain great advantages over the traditional focuses in Mechanical Engineering (ME) or MET with respect to the employment opportunities and salaries[5]. Therefore, robotics is becoming one of the most attractive majors in the Department of ME and MET. On the other hand, robotics (as shown in Fig. 1) is one of the most comprehensive majors since it needs to address many extremely complicated problems involving science, technology, engineering and math (STEM), for example, algebra, ME, EE, CS, and IT[6]. In addition, some of the sub- topics related to STEM are also listed in Fig. 1. Hence, the students in robotics programs must master intensive interdisciplinary knowledge of STEM if they want to succeed in their chosen program. However, the students of B.Tech program of robotics in MET are suffering three difficulties:

The weak fundamental knowledge of EE, CS and IT is the main obstruction[ 7 ]. In MET, the baccalaureate-level courses mainly focus on the mechanical system design, mechanics, dynamics, and simulation[ 8 ]. In addition, in reference [8], there are only two courses to introduce the mechatronics. Therefore, the students lack systematic training in the area of EE, CS and IT. It means that the instructors must go over the relevant knowledge in order to let students catch up with the requirements of the robotics classes.
One more challenge is to understand the advanced concepts and theories[9]. B.Tech. is different from the Bachelor of Science(B.S.). The program of B.Tech. mainly focuses on the hands-on skills while B.S. emphasizes theories and concepts [10, 11, 12]. One of the educational goals in MET is to cultivate future technologists rather than researchers. The curricula of MET emphasize on the applications, but the curricula of ME emphasize the theories. This difference obviously impairs the students’ deeper understanding of the advanced concepts and theories.
The limited class hours constrain the extended applications and furtherly result in the lack of opportunities to practice the knowledge of advanced robotics [13]. In addition, the limited class hours also make it difficult to go over the fundamental knowledge of EE, CS and IT listed in Fig. 1.

Fig. 1: Knowledge structure of robotics

In order to overcome the above problems, it is necessary to devise the appropriate courses for the B.Tech. program of robotics in the department of the MET while fully catering to the commitment of the college and the degree work of the BTech students [14].

II. HIERARCHICAL CURRICULA OF B.TECH. PROGRAM OF ROBOTICS

A. Current robotics curricula design and its limitation forB.Tech students

Although, the pre-college program of K-12 in some high schools opens robotics curricula to prepare the high school students for the successful transitions to postsecondary education and employment [15]. In this paper, only the robotics curricula for the B.Tech program of robotics are discussed. The robotics program is usually provided by EE, CS, or ME with major-oriented emphasis. In addition robotics curricula have been deployed at different levels of engineering and science education. In the college period, some institute takes robotics as an interdisciplinary engineering discipline[16]. Based on the commitments of the institutes and the students’ qualities, the degree paths should be different. Generally, fresh and sophomore are given the fundamental courses. For example, reference [17] introduced an implementation to migrate the components from the freshmen sequence into the sophomore engineering courses comprised of statics, circuits, and thermodynamics. Reference [18] used Lego Mindstorms kits to introduce the knowledge of robotics. Other implementations can be found in reference [19], reference [20]and reference [21]. Although these implementations are efficient for their programs, they are not appropriate for the junior classes of the robotics program in MET since the commercial robotics tools employed in these implementations prevent the students from practicing fundamental components. The senior-level robotics courses have been introduced in reference [22, 23, 24, 25, and 26]. In these kinds of literature, there are no exceptions to directly expose the students to the complicated control theories and algorithms. Certainly, this design method is reasonable for engineering students since they have more solid fundamental knowledge and stronger self-study ability. However, most of the students in the MET programs are struggling in mathematics classes. At the same time, they usually need to support themselves, and cannot totally devote themselves to the classwork. Therefore, how to utilize the limited class hours and how to integrate the knowledge into the practice become extremely critical.

B. Key components for successful curricula design

In order to introduce the curricular design, it is important to figure out the key components for the successful education, and then to examine the implementations based on these components. It is well known that education is one form of training. The training is the process designed to help a person to acquire knowledge, skills, and competencies. Ten key components for success training include (i) collaboration, (ii) starting with the end in mind (backward design), (iii) awareness of learning styles, (iv) use of a variety of learning strategies, (v) awareness of audience/relevance, (vi) facilitation of learning vs. pure instruction, (vii) creation of authentic learning, (viii) active participation, (ix) use of assessment tools and (x) evaluation (see Fig. 2)[27,28]. The best education and curricula design is to include all the components.

Fig. 2: Ten key components for the successful education

C. Structure of hierarchical curricula

The structure of the robotics curricula is gradual and hierarchical. The students must rise to the challenges of the more advanced courses while they keep progressing in their studies. Commonly, mastery of fundamental knowledge determines whether they can understand more sophisticated concepts. It is necessary for students to figure out how scientific theories are developed and what their limitations are, and then, to explore the potential applications of these theories[29]. Theoretically, the students who enrolled in the robotics classes are assumed having the fundamental knowledge as shown in Fig. 1 and Table I. when they finish the fourth semester[30, 31, 32, 33, 34, 35, 36]. Following that, higher-level courses are given. By integrating the theories into the practice, the students can overcome the difficulties of the abstract theories, and develop their own applications.

Therefore, the ideal arrangement of different levels of robotics courses should be in a scaffolding form and range from ‘introduction’, ‘application’ to ‘advanced’. The structure of hierarchical curricula can be found in Fig. 3. The courses at the introduction level are used to go over the fundamental knowledge, typical applications, usage of basic components, and embedded programming. These aspects, in fact, are very important for future study in the program of robotics. The other courses taken by students include “Quality Control”, “Advanced Solid Modeling”, “Simulation and Visualization”. After the specific training obtained at the first level of courses, the students have had commonsense and skills to go to the next level. At the application-level, the courses will focus on the exploration of the potential applications of the knowledge introduced in the introduction level while the theories are expanded. The aim of these courses is to consolidate the fundamental knowledge and sharpen the hands-on skills. In addition, the protocols of communication and interfaces will be discussed in detail. Following these, the students should have the ability to integrate the theories into some advanced applications, for example, the rescue robot, the smart building, and the robot arm. During this period, the students also take “Mechanical Measurements and Instrumentation”, “Advanced Strength of Materials”, and other flexible core courses (“Vibration and Advanced Dynamics” or “Finite Element Methods”). At the advanced level, the courses will introduce advanced control theories, advanced algorithms, and cognition in robotics. Therefore, the PID control, the algorithms of AI, and the knowledge of machine learning should be introduced and practiced. The anticipated outcomes of these courses will be that the students can solve the practice problems by themselves. In this level, many challengeable courses are provided simultaneously, which include “Project Management”, “Fluid Mechanics”, “Senior Design Project”, “Computer-Integrated Manufacturing”, and “Computer- Integrated Manufacturing”.

Table 1. Fundamental Knowledge of STEM in the First Two Years

*Math*	• Calculus: convolution • Differential equations: • Linear algebra: matrix and operation • Statistics: Bayes theorem
*Science*	• Chemistry: chemical sensor • Physics: thermal sensor • Liberal courses: logic thinking
*Engineering*	• Materials: processing and fabrication • Mechanics: failure theories • Dynamics: movement analysis • Drawing: PCB and structure drawing • Electric and electronics: impedance, signals processing
*Technology*	• Machining and Tooling: manufacturing methods and operation of machines • Soldering: ability to integrate components • Programming: embedded programming • CAD: design circuit and structure

Fig. 3: Structure of hierarchical curricula of robotics

III. DESIGN OF ROBOTICS CURRICULA

A. Courses of introduction level

The goal of the introductory course is to expose the students to robotics concepts, applications, and fundamental theories. Then, the students can deepen their understanding of the fundamental knowledge and familiarize themselves with the basic robotics techniques.

The anticipated students’ outcomes include: (1) Be able to gather, interpret, evaluate, and integrate information from a variety of sources. (2) Be able to communicate clearly and effectively. (3) Be able to function as an effective team member. (4) Gain or improve capabilities: (a) Apply the theoretical knowledge including the principles of analog and digital logic, concepts of digital logic analysis and design, microprocessor fundamentals and programming microcontrollers using C programming language. (b) Improve technical ability with an emphasis on the use of microcontrollers and programmable logic devices to interface with mechanical devices. (c) Using technology in conjunction with established theory to analyze mechatronic design problems.

Based on the key components of the successful education (refer to Fig. 2), the qualifications of the students should be taken into account, and this aspect corresponds to the component of ‘audience/relevance’. As discussed in the first part of this paper, the students of MET have three difficulties when they take the robotics courses. In order to overcome these difficulties and let the student well prepared for the challenges from the upcoming high levels of courses, the basic level was designed.

There is only one basic level of course. This course is named “Introduction to Embedded Systems Fundamentals and Applications in Robotics”. In this course, the students are required to design a simple robotics system composed of mechanical transmission, chassis, power source unit (PSU), control module, and the communication module (refer to Fig. 4). Through a series of design procedures, the students can master discipline-specific knowledge, skills, and tools. They can improve their capabilities with respect to the theoretical knowledge, the applications of hardware and software, the problem analysis, and the collaboration. In this course, the specific robotics system was a robot car called MazeBot. This was designed as a group project. The MazeBot was required to find out a path to escape from a maze. The project was decomposed into different parts based on Fig. 4, and the final design was assumed to complete step by step with the proceeding of the course. This method was supported by several components of successful education: ‘facilitation of learning’ (students take more control of their learning process by intensive hands-on experiments), ‘collaboration’ (group projects enhance the collaboration between each other), ‘learning style awareness’, ‘learning style variation’ (students are well known to the difference between the lecture-based and hands-on based classes. They enjoy the hybrid learning style, and are willing to be active during the project-based classwork) and ‘active participation’(The students are required to complete all the hands-on work by themselves, and are encouraged to expand the project into the practice).

Fig. 4: Structure chart of robotics system designed in class

At the beginning of this course, the embedded development kits (EDK) based on Arduino UNO R3[37] and the project of MazeBot were introduced. Then, the students were guided to go over and practice the fundamental knowledge of ME, EE, CS and IT through the EDK and the project. When the PSU was introduced, the analog circuit and corresponding components (resistor, capacitor, and inductor) were practiced in order to achieve a steady power source. When the EDK was employed, the fundamental knowledge of EE and CS were practiced which included the single-chip processor, peripheral circuit, serial communication, DA/AD converter, timer, logic, and embedded programming. For the part of sensors, only several simple sensors including thermosensor, IR sensor, and ultrasonic sensor were introduced in this course. For actuators, the relay, DC motor, and servo motor were introduced. The digital circuit and transistors were practiced during designing the control circuit of the motors. The Bluetooth-based wireless control was integrated into the MazeBot, and its function was to switch the power remotely via a smartphone[38]. All these works took 9 weeks. Although some new knowledge was discussed, the main goal of the above implementations was still to go over and apply the fundamental knowledge mastered in the first two years. Therefore, this process complied with the component of ‘backward design’. Following that, the chassis and transmission system of MazeBot were designed. In this part, the knowledge of machine design, CAD, and mechanics was employed. It took 2 weeks since the students in MET had a solid knowledge of ME[39]. Next, the students took 3 weeks to assemble the MazeBot, develop the algorithm to enable the MazeBot escaping from the maze, test the MazeBot, demonstrate their achievements, and present their projects. This 3-week time was a period of ‘evaluation’ and ‘assessment’. Hitherto, all ten components of successful education had been included in this project-based course. One of the MazeBots designed by students can be found in Fig. 5. This design employed the ultrasonic sensor, servo motor, DC motor, Bluetooth module, motor controller, Arduino EDK, the knowledge of CAD, programming, circuit design, algorithm design, and wireless communication. In addition, the appearance of Mecha Sonic demonstrated the students’ imagination and the solid fundamental knowledge of industrial design.

B. Courses of application level

In this level, the goal is to let students master the applications of popular sensors and actuators, practice various communication methods based on different protocols, and combine the fundamental knowledge, skills and simple projects into complicated projects.

Fig. 5: MazeBot with the appearance of Mecha Sonic.

The anticipated students’ outcomes include: (1) Be able to apply theoretical knowledge combined with the sensors & actuators into the project design. (2) Develop algorithms and programs to control the motion of simple mechatronic devices with the actuators and sensors. (3) Select appropriate devices for an embedded application to fabricate an applicable mechatronic system. (4) Improve the ability of collaboration.

The course name, at this level, is “Actuators and Sensors Application in Robotics”. In this course, an educational framework of wireless sensor and control network was employed. The Arduino MEGA 2560 board, sensor and actuator kit was selected to implement the hands-on experiments. The students were assigned to different projects. Finally, one large project named ‘smart building’ synthesized some of these projects. This large project fully employed the educational framework to realize remote access and control to the appliances of the building. More details about this course can be found in [39]. The final project can be found in Fig. 6 [39]. In addition, the design of this course also obeys the rule: the ten components of successful education should be involved.

Fig. 6: Project of smart building

C. Courses of advanced level

The courses of advanced level try to deliver and practice advanced knowledge involving hydraulic control, pneumatic control, PID control, and AI. There are two courses. One is “Control Systems in Robotics”. The other is “Robotic Systems Design and Applications”. The rule of the course design is the same as the former two levels. In the first course, the control techniques are emphasized. In the later one, the cutting-edge techniques of robotics are employed.

In the first course, the anticipated students’ outcomes include: (1) Demonstrate the knowledge of basic control theories used in the robotic systems. (2) Apply proper PID control techniques when designing smart devices or robotics that show proper dynamic behaviors. (3) Establish mathematical models based on electrical and mechanical principles. (4) Demonstrate the ability to communicate effectively and work as contributing partners for group projects.

In this course, the students were given the assignment of a project which must employ multiple control strategies and techniques involving hydraulic control, pneumatic control and PID control besides the skills gained through the courses of introduction level and application level. One of the projects named RescueBot in this course can be found in Fig. 7. As an unmanned vehicle, RescueBot was designed to clear the obstruction on the road. It was equipped with a gyroscope, three ultrasonic sensors, pneumatic transmission system, and pneumatic breaker. The technique of path management was employed to realize self-driving in which the path was planned and optimized by dealing with a straight path, circle path, and the combination of the two types of paths[40]. The pneumatic breaker was used to destroy and clear the obstructions.

Fig. 7: RescueBot

In the second course, the anticipated students’ outcomes include: (1) Be able to apply basic AI algorithms. (2) Use rapid prototyping techniques to fabricate components for their design projects based on the multidisciplinary design knowledge and skills. (3) Improve the ability of collaboration and prepare for future job seeking.

In this course, AI is introduced and combined with the skills from the other three courses. One of the applications of AI used here was object recognition based on vision and machine learning. During this course, the students were guided to study the basic concepts involving data training, data classification, decision tree, and search algorithm. Following that, they were instructed to use the machine learning module and object recognition module in OpenCV to develop their own applications. One of the projects was SocialBot which had the abilities of face recognition and speech recognition (refer to Fig. 8). SocialBot was composed of two Raspberry Pi boards to process and manage the data, two ‘OpenMV Cam M7’ cameras to acquire image information, one ‘ReSpeaker Mic Array’ mic array to realize speech recognition and one servo motor to rotate the head. For the details of the face recognition can be found in[41]. The method of speech recognition with ‘ReSpeaker Mic Array’ can be found in[42]. Since the limitation of the course hours, the mobility of the SocialBot was taken away, but the students enjoyed their achievement brought by the cutting-edge techniques.

IV. OPERATION, OUTCOMES, AND DISCUSSION

A. Operation

The robotics program has been operated for 5 years, but it is still a new concentration compared with other concentrations in the department. Hitherto, one round has been implemented. Only the students of MET are enrolled in this series of classes. In the first level, there are about 20 students including sophomore and junior each semester. During the implementation, most of the students are willing to continue the robotics classes. Therefore, in the second level, there are about 18 students each semester. For the advanced level of classes, they are given in different semesters to guarantee the number of students in each class (about 17 students per class). In addition, any student who has taken either course of the first level and second level are qualified to enroll in the advanced classes. All the courses are administrated by the instructors who take the responsibility of preparing the experiments. Moreover, a peer assessment mechanism is established to evaluate the performance of the students. This mechanism combines the instructors’ evaluation (70%) and the evaluation from other students in the class (30%) to decide the final individual grade. Besides these, the instructors will review the project report and make sure plagiarism-free.

The grading rubric used in class refers to Table II. This rubric uses a full 100 points to make a big difference in the credit products students can get. The students are credited based on the quality of their work with respect to each perspective. Then, the final scores will be the sum of all the sub-grads. In this table, the “Relevance” means that whether the projects use the knowledge from the class or the extension (papers or online materials) to the class. The classed are used to let the students learn and improve their hands-on skills, all the class-related materials can be applied in the project. Certainly, the plagiarism is definitely checked. Therefore, the students must provide appropriate references if they use extensive materials. For example, a robot car in the introduction level class has the function of facial recognition. The grade will be lower than others if the performance of the motor control is weaker than others no matter how their vision-based work is perfect or not because the vision-based work is taken as the weak-connected knowledge. To the “Contribution”, this is used to encourage the students to integrate the knowledge into practical applications. Therefore, the students must propose their projects and explicit the specific application of their project in the real world. “Innovation” mainly shows the students’ ability to employ new techniques to develop their projects. For example, the application of Bluetooth techniques in the application level of course. The “Technological Sophistication” is taken as a comprehensive indicator to evaluate whether the students demonstrate the ability to logically synthesize the in-class knowledge and the extension together instead of a simple overlap. For instance, the multiple sensed data from different sensors must be furtherly processed and used to help to make control decisions if the least square root method is employed in the advanced level of course. To the “Economic efficiency”, the students must articulate that their project is created based on the cost-manufacturing graph. Through the performances in these projects, the anticipated outcomes of the students can be inspected thoroughly.

Fig. 8: SocialBot

Table 2. Grading Rubric

STANDARD	FULL SCORE
Relevance: Rate how the project is relevant to the class	20
Contribution: Rate how this project makes a contribution to practical applications	20
Innovation: Rate how this project applies new ideas into the design	20
Technological Sophistication: Rate how this project is sophisticated	20
Economic efficiency: Rate how this project cost	20

B. Outcomes

Through the 5-year operation, most of the students have satisfied performance in their class projects. Certainly, the important evaluation standards of the courses are (1) the improvements in respect to the knowledge and the skills, (2) the students’ career prospects.

For the first standard, the in-class projects have shown that the students can master and apply the interdisciplinary knowledge, can solve the practical problems, can explore the potential applications of the fundamental theories, can employ modern design tools, and can demonstrate the creativity. In the project-based courses, the students gradually increase their capabilities following the scaffolding structure of the courses. The final scores of the students are composed of 10% for the participation in class including attendance, behaviors, motivations; 20% for the homework; 25% for the midterm and 45% for the projects. After completing the projects, the final scores are graded, and the final scores range from 82 to 96. In addition, the pre-class and post-class surveys for the specific skills show that the students have more confidence in their knowledge after the classes (refer to Table III). Moreover, the students’ performances are evaluated based on the requirement of the anticipated outcomes. Through both the self-evaluation and the instructors’ evaluation, the students have proved that they are well prepared for future challenges with respect to the knowledge and skills in the area of robotics.

Table 3. Skills Self Evaluation Survey

In order to show the students’ achievements in class, the project grades of all enrolled students in each course are collected and analyzed. At the same time, it is assumed that the student meets the least requirement of the course if his or her grade is better than 80. So, if this assumption is taken as a statistical hypothesis, and the corresponding p-value or level of significance is 5% (left-tail event), the observed data (student grade) have a higher significance. Based on the collected data, the p-value can be obtained. Therefore, the statistical results are shown in (a) through (d) are reasonable. In these 4 diagrams, the total students counted are 194 for introduction level, 176 for application level, 138 for advanced level-1, and 114 for advanced level-2. The relatively lower number in advanced levels results from the availability of these two courses since they are not provided simultaneously. In addition, with the increase in the difficulty, the quality of the projects has a little bit of decline. Therefore, the grade of the project is also declined with the increment of the course level. The main reason for this trend is that the projects are more and more complicated, and the probability of malfunction is increased accordingly during the demonstration of the prototype, especially in the advanced level-2 classes. One more thing is that the standard error in Fig. 9 (d) is a little larger than others, and it means that the performance of the students tends to diverge due to the polarization of the project integrity.

Besides the project grade analysis, an assessment survey is administered at the conclusion of each course. The questions of the survey and average scores (on a scale of 1 to 5) of the answers are listed in Table IV. This survey shows that the students in the robotics classes have gained great confidence by taking the courses, developing the projects and employing modern design tools. Furthermore, the students also have a strong interest in the robotics classes, and they are willing to recommend their classes to others. Hence, the designed curricula are successful with respect to the students’ feedback.

For the second standard, statistics about the internship and the fulltime positions received by the students of the robotics program of MET is made. The results of the statistics show that 78% of the junior students can get offers of internship of robotics, and 87% of the graduates can find full-time jobs related to robotics once graduating. These results are much better than other traditional concentrations of ME/MET (67% and 71% separately). Therefore, both the students’ performance and their career prospects prove that the robotics courses are helpful to operate the robotics program in the department of MET.

C. Discussion

One of the main challenges during designing the curricula is to let the students familiarize themselves with the robotics system, the design procedures and the practical applications of the fundamental knowledge while avoiding to expose to the complicated theories. In order to overcome this challenge, the classwork mainly focuses to guide the students to learn the catalogs of components used in the common robotics systems. Then, they are guided to learn the how to read, understand and use manufacturing datasheet to find out the critical parameters of the components, for example, the voltage, current, and outline drawing of a capacitor, or the input, output, logic, and recommendation peripheral circuit for an ATmega2560 CPU. After that, the students directly use the ready to use an experimental platform to test the performance of the components instead of analytical calculations. For example, in the application level classes, the students are given a wireless sensor and control network framework. The students are required to learn the data acquisition, the usage of sensors, the application of the communication protocols and the motor controllers. After that, the students do not need to immerse themselves in complicated algorithms with respect to data processing.

Figure 9. Project grade statistical result of four courses

(a) Introduction Level

(b) Application Level

(d) Advanced Level-2

On the other hand, the students learn how to use electrical computer design software (EAD) to design and simulate the circuit, and how to use the dynamics and kinematic analysis software to simulate a robotic mechanism. By the employment of the EAD software, the students will not directly learn how to operate the registers, how to analyze a complicated circuit, how to deal with the break, and how to design the data processing algorithms. By the application of the dynamics and kinematic analysis software, the students can adjust the links, the masses, the joints, and freedoms without the concern of the complex operations of matrix and differential equations.

Therefore, both the contents of hardware and software facilitate the students to succeed in class without evolving in pure theoretical analysis. It is worth mentioning that the graduate of B.Tech. are often engaged in the jobs of marketing and maintenance. These jobs highly require a wide horizon of knowledge and plenty of practical experiences. The designed curricula can provide a chance to the students to learn common and up-to-date robotics systems and components while enhancing their hand-on ability by the class projects, and then, let them competitive and well prepared for their future careers.

Table 4. Post-Class Survey Results

V. CONCLUSIONS AND FUTURE WORK

In this paper, the design of curricula for the robotics program of MET is introduced. This work is the most critical part of a hands-on-project-based program. In order to explain why the curricula design is necessary, the difficulties faced by students of the robotics program of MET are discussed. Then, the hierarchical curricula in the robotics program are presented after a series of discussions including a survey of the robotics curricula, an introduction of the component of successful education, and the structure of hierarchical curricula. Following that, the scaffolding structure curricula are designed, which include introduction level, application level, and advanced level. The introduction level is to combine the fundamental STEM knowledge together to enable students solving practical problems. The application level is to guide the students to expand the application of the theories. The advanced level is to facilitate the students to integrate the cutting-edge techniques into practical applications. It has been proved that the designed curricula can enhance the students’ understanding of the fundamental concepts, can inspire the students’ interest in robotics, and can improve the students’ performance. In the future, the robotics curricula will be optimized further to adapt to the development of society. In addition, more up-to-date techniques will be introduced and applied in-class projects.

ACKNOWLEDGEMENT

The authors wish to thank the students of the robotics program in CUNY New York City College of Technology.

REFERENCES

[1] Türk, D.A., Triebe, L. & Meboldt, M., 2016, “Combining additive manufacturing with advanced composites for highly integrated robotic structures”, Procedia CIRP, Vol. 50, pp 402-407.

[2] Hamblen, J.O. & Van Bekkum, G.M., 2013, “An embedded systems laboratory to support rapid prototyping of robotics and the internet of things”, IEEE Transaction of Education, Vo. 56, No. 1, pp. 121-128.

[3] Brady, M., 1984, “Artificial intelligence and robotics”, In Robotics and Artificial Intelligence, pp. 47-63, Springer, Berlin, Heidelberg.

[4] Zhang, Z., Zhang, M., Chang, Y., Aziz, E.-S., Esche, S. K. & Chassapis, C., 2018, “Collaborative virtual laboratory environments with hardware in the loop”, In Cyber-Physical Laboratories in Engineering and Science Education, Springer, pp. 363-401.

[5] Wisskirchen, G., Biacabe, B.T., Bormann, U., Muntz, A., Niehaus, G., Soler, G.J. & von Brauchitsch, B., 2017, “Artificial intelligence and robotics and their impact on the workplace”, IBA Global Employment Institute.

[6] Yang, G.Z., Bellingham, J., Dupont, P.E., Fischer, P., Floridi, L., Full, R., Jacobstein, N., Kumar, V., McNutt, M., Merrifield, R. & Nelson, B.J., 2018, “The grand challenges of science robotics”, Science Robotics, Vol. 3, No. 14, pp. eaar7650-7664.

[7] Stern, F., Xing, T., Muste, M., Yarbrough, D., Rothmayer, A., Rajagopalan, G., Caughey, D., Bhaskaran, R., Smith, S. & Hutchings, B., 2003, Integration of simulation Technology into undergraduate engineering courses and laboratories”, Proceedings of ASEE 2003 Annual Conference, Nashville, Tennessee, USA, June 22-25, 2003.

[8] Jovanovic, V., Verma, A. & Tomovic, M.M., 2013, “Development of courses in mechatronics and mechatronic system design within the mechanical engineering technology program”, Proceedings of the 11th Latin American and Caribbean Conference for Engineering and Technology, 14 – 16 August, 2013, Boca Raton, Florida, USA.

[9] Jovanovic, V.M., Michaeli, J.G., Popescu, O., Moustafa, M.R., Tomovic, M., Verma, A.K. & Lin, C.Y., 2014, “Implementing mechatronics design methodology in mechanical engineering technology senior design projects at the Old Dominion University, Proceeding of 121st ASEE Annual Conference & Exposition, Indianapolis, Indiana, USA, June 15-18, 2014.

[10] https://floridapoly.edu/difference-engineering-technology-degree- bachelor-science-engineering/, accessed in February 2020.

[11] Gqibani, S., Clarke, N. & Nel, A.L., 2018, “The order of skills development for technician and technologist training curricula”, Proceedings of 2018 IEEE Global Engineering Education Conference, Santa Cruz de Tenerife, Canary Islands, Spain, April 18-20, 2018.

[12] Zhang, Z., Zhang, A.S., Zhang, M. and Esche, S.K., 2019, “Design and application of a platform of wireless sensor and control network in robotics course of mechanical engineering technology”, Computers in Education Journal, Vol. 10, No. 1, pp. 1 – 7.

[13] Cheng, H., Hao, L., Luo, Z. & Wang, F., 2016, “Establishing the connection between control theory education and application: an Arduino based rapid control prototyping approach, International Journal of Learning and Teaching, Vol. 2, No. 1, pp. 67-72.

[14] http://www.citytech.cuny.edu/catalog/docs/catalog.pdf, accessed in February 2020

[15] Karim, M.E., Lemaignan, S. & Mondada, F., 2015, “A review: Can robots reshape K-12 STEM education”, 2015 IEEE international workshop on Advanced robotics and its social impacts (ARSO) (pp. 1- 8), IEEE.

[16] Gennert, M.A. & Putnam, C.B., 2018, “Robotics as an Undergraduate Major: 10 Years’ Experience”, Proceedings of 125th ASEE Annual Conference & Exposition, Salt Lake City, UT, USA, June 24-27, 2018.

[17] Harbour, D. & Hummel, P., 2010, “Migration of a robotics platform from a freshman introduction to engineering course sequence to a sophomore circuits course, Proceedings of Frontiers in Education Conference, Arlington, Virginia, USA, 27 – 30 October 2010.

[18] Tester, J.T., 2008, “Management of a large team-design and robotics- oriented sophomore design class”, Proceedings of 38th ASEE/IEEE Frontiers in Education Conference, October 22 – 25, 2008, Saratoga Springs, NY, USA.

[19] Azemi, A. & Esparragoza, I., 2005, “Problem-based collaborative projects in and between freshman and sophomore engineering courses”, Proceedings of ASME 2005 International Mechanical Engineering Congress and Exposition, Orlando, Florida, USA, November 5 – 11, 2005.

[20] Zhang, M., Duan, P., Zhang, Z. and Esche, S.K., 2018, “Development of telepresence teaching robots with social capabilities”, Proceeding of ASME International Mechanical Engineering Conference & Exposition IMECE’18, Pittsburgh, PA, USA. November 9-15, 2018.

[21] Momeni, A., Previlon, F., Despopoulos, A., Schirner, G., Kimani, J. & Kaeli, D., 2015, “Engaging sophomores in embedded design using robotics”, Proceedings of the Workshop on Computer Architecture Education, Portland, OR, USA, June 13, 2015.

[22] Mataric, M.J., 2004, “Robotics education for all ages”, Proceedings of AAAI Spring Symposium on Accessible, Hands-on AI and Robotics Education, Stanford University, Palo Alto, California, USA, March 22– 24, 2004

[23] Correll, N., Wing, R. & Coleman, D., 2013, “A one-year introductory robotics curriculum for computer science upperclassmen”, IEEE Transactions on Education, Vol. 56, No. 1, pp. 54 – 60.

[24] Cappelleri, D.J. & Vitoroulis, N., 2013, “The robotic decathlon: Project- based learning labs and curriculum design for an introductory robotics course”, IEEE Transactions on Education, Vo. 56, No. 1, pp. 73 – 81.

[25] Ciaraldi, M., Cobb, E., Looft, F., Norton, R. & Padir, T., 2009, “Designing an undergraduate robotics engineering curriculum: Unified robotics I and II”, Proceedings of the 2009 ASEE Annual Conference & Exposition, Austin, Texas, USA, June 14 – 17, 2009.

[26] Padir, T., Gennert, M.A., Fischer, G., Michalson, W.R. & Cobb, E.C., 2010, “Implementation of an undergraduate robotics engineering curriculum”, Computers in Education Journal, Vo. 1, No. 3, pp. 92-101.

[27] http://www.pages.drexel.edu/~ctc27/tenkey.html, accessed in February 2020.

[28] Chang, Y., Aziz, E.-S., Zhang, Z., Zhang, M. & Esche, S. K., 2016, “Usability evaluation of a virtual educational laboratory platform”, Computers in Education Journal, Vol. 7, No. 1, pp. 24-36.

[29] Danielson, S., Hawks, V. & Hartin, J.R., 2006, “Engineering technology education in an era of globalization”, Proceedings of the 36th Frontiers in Education Conference, San Diego, California, USA, October 28-31, 2006.

[30] Worcester Polytechnic Institute, “Undergraduate catalog 2018 – 2019”, Wpi.edu.

[31] New York City College of Technology, “Catalog spring 2019”, Citytech.cuny.edu.

[32] University of California Santa Cruz, “Robotics engineering B.S. degree 2017-2018 curriculum chart”, Ucsc.edu

[33] University of Michigan Dearborn, “Robotics engineering core curriculum 2018”, Umdearborn.edu.

[34] Lawrence Technological University, “Bachelor of science in robotics engineering (BSRE) flowchart, Fall 2017”, Ltu.edu.

[35] Central Conneticut State University, “Bachelor of science in robotics and mechatronics engineering technology program”, Ccsu.edu.

[36] Purdue University, “2018 – 2019 University catalog: robotics engineering technology, BS”, Purdue.edu.

[37] Barrett, S.F., 2013, “Arduino microcontroller processing for everyone!”, Synthesis Lectures on Digital Circuits and Systems, Vol. 8, No. 4, pp. 1- 513.

[38] https://engineersportal.com/blexar, accessed in February 2020.

[39] Zhang, Z., Zhang, A.S., Zhang, M. & Esche, S.K., 2018, “Conceptual framework for integrating a wireless sensor and control network into a robotics course for senior students of mechanical engineering technology”, Proceedings of the 2018 ASEE Annual Conference & Exposition, Salt Lake City, UT, USA, June 24-27, 2018.

[40] Liu, Y. & Bucknall, R., 2015, “Path planning algorithm for unmanned surface vehicle formations in a practical maritime environment”, Ocean Engineering, Vol. 97, pp. 126-144.

[41] Zhang, Z., Aziz, E.S., Esche, S. & Chassapis, C., 2018, “A virtual proctor with biometric authentication for facilitating distance education”, In Online Engineering & Internet of Things, pp. 110-124, Springer, Cham.

[42] http://wiki.seeedstudio.com/ReSpeaker_Mic_Array_v2.0/, accessed in February 2020.

The post Project-Based Courses for B.Tech. Program of Robotics in Mechanical Engineering Technology appeared first on ASEE Computers in Education Journal.

Improving Student Success by Being Automatically Personal

Francis Igot — Sun, 05 Jan 2020 21:44:16 +0000

Mark A. Palmer

Formerly IME Department
Kettering University
Currently Flushing, MI
MarkAPalmer@att.net

Abstract – This paper describes the development and use of “automatically-personal e-mail” routines allowing one to send interpretive e-mails to one’s class based on clicking a command in an Excel grade book. The macros are included in a template file which are available under the Attribution-NonCommercial-ShareAlike 4.0 International Creative Commons License. Nudges, in the form of light-touch directed-feedback have been shown to be effective in engaging students, but they are often time consuming for faculty. The author has found that he can send detailed performance updates to students automatically through macros in a well defined Excel Gradebook. This increases student engagement as they see it as a way of demonstrating caring. Using an Engineering Materials Course as an example, the author demonstrates the steps necessary to send 6 such nudges throughout an 11 week term. Sample commented coding and examples of the messages sent to students are provided as examples. Students are very satisfied with the auto-emails as shown. The paper closes with an analysis meant to help others implement this system in the minimal amount of time.

I. NEED

A. What is a Nudge

A “nudge,” is an intervention that encourages, but does not mandate, a certain behavior [1]. One form is “Light-touch, targeted feedback” sent to students via email can improve their perceptions of and performance in a class [2]. A professor sent personalized supportive e-mail to students to students who failed the first exam and found a small investment of time, signaled students that she cared and believes boosted recipients’ performance in the course [1]. Nudges are really demonstrations of caring which result in student engagement. Nudges are the answer to the question raised by another author who asks how do we engage the significant fraction of students who struggle and are often forgotten [3]. Reaching out can be complicated and time-consuming for the professor; the person most effective at nudging students to ask for help. In the author’s opinion all students should be nudged frequently throughout the term. This requires responding to another author’s question: What if engagement wasn’t complicated and didn’t take that much time?2. Automatically personal e-mails can be effective light-touch targeted-feedback nudges which engage students.

Student engagement in response to nudges builds resiliency. Currently, higher education institutions are facing a crisis with declining enrollment and student attrition. … Resiliency is one factor which will impact a student’s decision to stay or go [4]. Dictionary.com’s definition of resiliency is: ability to recover readily from adversity defined as an unfortunate event. The unfortunate event can be a lower than expected grade or expectation thereof. We can help students recover, by helping students be resilient, through increased engagement initiated by light-touch targeted-feedback nudges. These nudges designed to help students respond to difficulties in the classroom is most effective when specific feedback has been provided to the student by the professor so that key issues can be pinpointed and addressed [4]. I have been automatically personal with students for years. I began (early 90’s) with handwritten notes on each of 3 tests, moved to fill-in-the blanks on print, then to VBA which allowed for six nudges per term. Each message is feedback and makes it easy for the student or others helping the student pinpoint issues.

B. How this Nudge Works

In one example a professor started sending personalized emails (in which the body of each email was the same, but they were addressed to each student individually) to students who received a failing score on the first of three exams—approximately 10 percent of a class of 200. The email, 1) pointed out that the student didn’t do as well as expected on the exam, however, it was still early in the semester, and that changing habits now could turn their grade around. 2) asked the student whether they knew why they hadn’t performed well. 3) gave some suggestions and 4) finished by reiterating the fact that together we could improve their standing in the class [3]. Committing to doing the same is committing to a complicated time-consuming process. The automatically personal e-mails sent out to the entire class included (1), (3) and (4) and were not time-consuming. The Excel gradebook communicated with Outlook to nudge students through e-mail.

C. Results of a Nudge

The only negative feedback from students in the articles cited came when a message was sent out in error. “I was wondering why I was sent this message,” [a welcome-style email] one student wrote to a participating art instructor. “I believe that I have been coming to class regularly as well as taking notes actively. Thank you for your time.”(Kurlaender and Carrell)2. Linking the feedback to an Excel Workbook could have solved this problem. Also it would have been a chance to praise the student. Initial praise is as important as reaching out to those who are struggling.

Student response to nudges were usually positive. “No professor has ever cared about me.” was a sentence in the reply sent by a struggling student. This shows that by identifying struggling students and sending them personalized emails encouraging action and providing support, educators can make a significant difference to the success of their learners. Sent 20 e-mails and gave 5 responses from students. All thanked her. 2 specifically cited appreciation for caring compared to other professors. 3 promised to reflect and contact her 1 of these 3 asked for a response3. A number of professors said they were surprised by their students’ gratitude at their gestures. For example “I attend every class, go to the review sessions, and have turned in the extra credit so I am defiantly [sic] trying to do well but I am still struggling, I will come to office hours and try to meet up with our TA as well. Let me know if there is anything else I can do. Thank you!” (Source: Kurlaender and Carrell) [2]. Another professor said, more than half of the 20 students she emailed wrote back expressing their appreciation for her message and taking responsibility for their grades1. This is why early nudges are important.

Some faculty were neutral regarding nudges.

Kurlaender and her co-author on the project, Scott Carrell, professor of economics and co-faculty director of the California Education Laboratory at Davis, wanted to see what would happen if professors reached out to students individually via email just a few times a term, with the goal of promoting their sense of self-efficacy and help-seeking behavior. Over all, they found strong evidence that this “”light-touch, targeted feedback”” can positively affect student perceptions about a course and instructor. After the first email “nudge” about homework, students in the Davis study increased the time they spent on homework. After a second nudge about performance on their first course exam, students scored significantly higher on a second exam, compared to students in a control group who did not receive the emails. exception: There was a significant positive effect on overall course performance for students with fewer than 30 previous college credits. There was no significant positive effect on course withdrawals [2].

I did 3 nudges similar to their 2 and noted similar increases in course performance (HWs and Tests). In some terms I noticed an increase in withdraws when students were informed once 26.5% of their grade was established.

Some faculty responded negatively regarding the use of nudges. There were a series of negative comments regarding this article [2].

This sounds like a ton of work for something described as “”light-touch””.
All this to produce “”no significant positive effect on overall course performance and I was disappointed to see the tepid results the study got on class performance.
No significant positive effect on course withdrawals,
Having faculty members provide some simple, individualized feedback to students as an intervention.” Done! I like to call them exam scores or quiz scores – it is individualized with notations about student errors and misconceptions and is provided early in the semester so that students might alter their trajectory.
Doing these nudges one at a time is likely impractical. For example, one faculty member commented they would need to send 564 emails per term based on a 4-4 schedule, an average of 47 students per class, and 3 emails per semester. I’d end up sending dozens of different varieties of emails to those students. The first, “”welcome”” email would require me to look at attendance for the first couple of weeks. Then I’d send either generic “”welcome”” or a “”welcome, but why aren’t I seeing you regularly”” email. That would be pretty easy. But, the second and third would require a complete analysis of attendance/exam/quiz/homework performance to be able to send the appropriate email. I could write an Excel process to automate this analysis and, if research can demonstrate a clear, substantial, significant benefit to a “”light touch”” then maybe I’d spend the time doing it. But not until

There is no doubt that well graded and corrected tests are a form of individualized feedback, however just providing such feedback without improvement suggestions should not be expected to help students alter their trajectory. It is clear that this type of feedback is most needed in large courses, and therefore having the instructor sit next to a computer with a bunch of grades and attendance records deciding which students get a bcc is probably not worthwhile.

Other faculty had positive reactions regarding nudges. One person said: This is an excellent idea. Without an email from a professor a student would have no way of knowing that they’re doing badly on assignments. One respondent thought it was sarcasm and another claimed that because faculty post grades to an LMS and return work with a grade to students they know how they are doing in the course [2]. I hope this is not sarcasm because there is no reason to leave the grade interpretation up to the student. From my experience LMSs do not provide interpretation, they simply supply a set of numbers.

Nudging each student throughout the term will not be practical unless automated. I did not find anything in the literature regarding automatic light-touch targeted-feedback nudges which engage students.

II. APPROACH

I created an automated system whereby once grades are entered in an Excel Workbook all students can be e-mailed at the touch of a button. I found the code shown in Text Box 1 through a Google search. This common code can be copied from macro to macro as one creates more automatically personal messages. The main research question to be answered is: what is
needed to make this method successful?

Text Box 1: Generic Code to Send e-mail from Excel

A. ID Points in Term

It is most likely both impractical and not beneficial to send an interpretive message after every grade has been recorded. My goal is to be most effective at improving student learning without generating what I describe as stock market watching pressure. Therefore the nudge points need to be based on the course structure. Consider the following course which meets on Mondays and Thursdays each week. There were 8 HW’s 10% (1-6 1% ea, 7 and 8 2% each) due Monday and 8 Tests 60% (7.5% each) given on the following Thursday. I chose to give 6 nudges.

1&2) Because HW is key to successful completion of the course, I felt it important to give interpretive feedback after HW 1 and HW 2.

3) After the students have had 3 HW assignments and 3 tests I send out the MidTerm 1 update. At this point in time I can relate HW and Test performance as well as ensuring that students are given an indication of their improvement by the time official midterm grades are posted.

4&5) Rather than merely posting an official midterm grade I send out an interpretive message the Midterm 2 update. Both the MT2 and pre-final updates include a detailed description of what students can accomplish.

6) The final notice is sent to students prior to the final grade being posted. I thank them for being part of the course, give them information so they can improve in future courses, and offer my help in the future. Also, if there is a mistake I can correct it before posting final grades and having to go through a grade change.

B. HW 1 Samples and Student Responses

Thank you for turning in an excellent HW Assignment. I appreciate all your hard work. Turning in assignments such as this will help ensure your success in this course. Thanks again and let me know how I can help you continue to excel

Often a thank you for caring.

Thank you for turning in a good HW Assignment. I appreciate all your hard work. The difference between a Good HW and an Excellent HW is usually a minor misunderstanding of the criteria. Please review them and contact me if you have any questions. Thanks again, let me know how I can help you continue to excel, and I look forward to correcting your second HW assignment. Remember Turning in assignments such as this will help ensure your success in this course.

I wanted to seriously thank you for going over the work in detail (notes, this bit of mail, etc), especially when I turned it in under an unacceptable format. I really hope that I am able to improve my test scores so I am not so worried about my grade and so I can really take the class for all that I can. I really enjoy working in my shop and everything that we have touched on so far is and will be directly applicable to my job. Also, thanks for taking the time to listen to me. I forget how much it can help to sit down with someone and go over questions and concerns. Although it might not have seemed like much, but it kind of gave me the chance to step back and look at my situation. Don’t be surprised if I show up early for lab more often, it seemed to help me this time and I will take whatever chance I can to improve myself. Have a good evening and see you tomorrow.

I noticed that the first HW you turned in was marginal. This concerns me because completing HW assignments is necessary to succeed in this course. Please review the first HW assignment and the grading criteria. Contact me if you have any questions, I realize that it might be getting used to the requirements. I want to help you succeed in this course. Thanks again and I look forward to grading your second HW assignment.

Thank you for your concern. I didn’t realize that when you asked for the memorandum that you wanted the whole lab report. I fixed that after the first homework. We have looked at your notes regarding our first homework assignment. Our team was wondering if there would be any way for us to redo the first homework assignment and receive partial credit back. If this is not possible, could we have you review homework assignment two to check for correct formatting? Thank you for the heads-up about the homework. I will certainly use your feedback in editing this week’s homework/lab report. Was there anything outside of the criteria sheet and commented homework that was of particular concern?

After reviewing the grades for the first HW assignment I noticed that the assignment you submitted was unacceptable. I am very concerned about this. Succeeding in this course requires that you submit acceptable assignments. Please let me know how I can help you succeed in this course. I look forward to: hearing from you, working with you and correcting your second HW assignment.

Because this is a group homework assignment, and all future assignments will also be done in this group we would all like to meet with you at the same time. Could we possibly meet with you after class this afternoon?

I noticed that you did not turn in the first HW assignment. I am very concerned about this because submitting acceptable assignments is key to your success in this course. Please contact me so that we can work together to ensure you complete the course successfully. I look forward to hearing from you, working with you and grading your second HW assignment.

Can we talk after class today and figure out what time works best to sit down and discuss your course.

C. HW 2 Sample

I felt it very important to follow up the first HW message with a second comparing the two. This allows me to congratulate students for progress made, nip possible decreases in performance in the bud, praise students with consistently high performance, and warn those whose HW performance is consistently less than acceptable. There are 11 possible messages based upon the combination of the 2 HW grades. These are based on 1 of 5 sublines, 1 of 11 gradelines and 1 of 5 interplines. The details of the gradeline(1- 11) and interpline(1-5) are in the code. A sample of code used for the HW 2 update is shown in Text Box 2.

Text Box 2: Case-Select Structure Sample for HW 2 Update

An example message (see Text Box 1) is below. emailmessage = salutation + Chr(10) + gradestate + ” ” + gradeline + ” ” + interpline + Chr(10) + Chr(10) + sigline

The sigline is standard. The salutation is based on the student’s name as shown in the code in Text Box 3. The gradestate, gradeline, and interpline are determined for each student from the data in the Excel Workbook. Chr(10) is needed for a hard return and “ ” is required for spaces

This is a sample message.
Dear WhoEver:

Text Box 4: Sample MidTerm 1 Message

Your first two homework grades are, for HW1: Unacceptable and for HW2: Marginal. Your performance on the HW has improved, thank you for the increased effort. However, it is very important that you turn in acceptable (Good or Excellent) HW’s Otherwise you will have difficulty successfully completing this course. Please contact me so I can help you continue to improve and thus succeed. I look forward to working with you.

D. MidTerm 1

MT 1 is sent out after I have 3 tests and 3 HW to relate. This also allows me to give students an indication of their improvement by the official midterm notification. An example message is shown I Text Box 4.

In order to determine the HW Message one must count the number of acceptable, barely acceptable, unacceptable, and skipped HW assignments as shown in Figure 1 (Excel).

Figure 1: Determining HW Status After 3 HW’s and 3 Tests for MidTerm 1 Update

E. MidTerm2 and Advance

Text Box 5 is a sample message sent at the same time the official midterm grade is sent to the registrar.

It is possible that a line of the Advance (pre-final email) might include the following. To earn a D in this course your score on the final exam will need to be at least -15. of 300 points. Students like knowing that they are guaranteed a grade higher than D or in some cases C.

Text Box 5: MidTerm 2 Message

F. Notice

A sample final notice message follows.

I wanted to let you know that your score on the final examination was 174, which corresponds to a D. As a result your course grade is: C+. You showed major improvement throughout the term, but your performance significantly declined on the final examination. Please review what you did during the term so that you can repeat it in future courses. Please assess how you prepared for the final examination, so that you can learn to use final exams to your advantage. Thank you for being part of this class. If I can help you implement the suggestions I made, or in any other way in the future, please let me know.

III. NET BENEFITS

A. Tipping Points

One must schedule time at tipping points. In the courses I teach all HW was submitted, graded, and corrected electronically. Thus HW can be returned outside of class time. Further HW was not graded based on correctness but on how easy it was for me to correct. Just before the HW is due I post narrative solutions. Consider the course shown earlier which has class-sessions on MTh and labs TW. HW is due each Monday. Thus if HW 1 is due on 2^nd Monday, it must be returned by 2^nd Friday.

Each HW takes approximately 10 minutes to grade and more importantly correct. If HW is assigned to 3 person teams and there are 48 students in a given class section this can be done in just under 3 hours. Note once the grades are entered in an Excel Workbook e-mails to all 48 students are sent out in less than a minute. My experience is that after sending out these e-mails one will get between 16 and 32 e- mails from students. as shown in Table (Messages v Responses) reading and responding will take between 1 and 2 hours. These 1 or 2 hours are rewarding as the 2-3 hrs time spent correcting not simply grading HW. This is not a sarcastic comment, I found the level of dialog to be intellectually rewarding. Scheduling time at tipping points is easy to do using a calendaring system. I suggest doing it at the beginning of the term. For example 2^nd Friday Priority Item – Grade HW 1, send out AutoPersonal Update 1/6, and work with students.

B. Pedagogical Survey

Students appreciate the automatically personal e-mails. I developed a pedagogical survey which was based on an ascending survey. Students were asked the following question.

To what extent did you find the automatically personal updates [HW 1, HW 2, Test 3 (MT 1), Test 5(MT 2) and Advance Grade] valuable?

The content of the automatically personal updates were useless at best. They may have hampered my ability to successfully complete the course. I do not even know if I will read the automatically personal final grade notification. (Unacceptable)
The content of the automatically personal updates were somewhat useful. However, they had essentially no impact on my being able to successfully complete the course. The automatically personal final grade notification will be nice to receive. (Marginal)
The content of the automatically personal updates were useful. I appreciated them, and they had some impact on my being able to successfully complete the course. The automatically personal final grade notification will be nice to receive. (Acceptable)
The content of the automatically personal updates were useful. I appreciated them. They positively impacted my being able to successfully complete the course. I look forward to the automatically personal final grade notification. (Good)
The content of the automatically personal updates were useful. I appreciated them. They had a major impact in my being able to successfully complete the course. I look forward to the automatically personal final grade notification. (Outstanding)

Figure 2: Student Satisfaction with Auto-Personal e-mails as a Pedagogical Technique. GB = Good or Better (good or outstanding), LTA = Less than Acceptable (unacceptable or marginal)

The student responses are shown in Figure 2. The criterion for each overall rating is indicated on the graph. The rating of acceptable for W15 and U16 were very close to good. In one case the %LTA was just above 25% and in the other the average was 2.6 instead of 2.7. The low ratings in S14 and F15 were do to very heavy teaching loads and hospitalizations which delayed grading. Students like the auto-personal e-mails, they expect them and are justifiably upset when they are not delivered. They cannot be delivered if one falls behind in grading.

C. Analysis

In about half of the terms mentioned in this paper there was an increase in W’s. This usually occurred after the third update. Students chose to give up even though 74.5% of their grade was undecided. I have every reason to believe that the % of W’s would decrease if the early performance updates were eliminated. Obviously, that is not an acceptable fix and should call attention to the use of DFW% as an inaccurate performance metric. In the future these e-mails will include what students need to do to achieve certain letter grades.
There is a chance that students when learning that very little effort is required for a D settle. This course was taken as the last sole course by a significant number of students. A student with a GPA of 3.052 or higher is guaranteed a 3.0 average if they pass the one course. 3.565 guarantees a 3.5 average.(Based on 40 courses and this being last remaining).
The coding can be done in Word as it is based on a lot of repetition.
The set up of the Excel Workbook is based on a lot of copy/paste.
The time spent interacting with students usually electronically after these updates are sent is extremely rewarding. Scheduling such times in advance, around when students are expected to respond is key as is scheduling time to grade when the checkpoint is due.
Once the macros are written, selecting a menu item after recording grades is not a lot of work and every student gets an automatically personal e-mail.
It is hoped that the macros, and workbook are a template that will minimize the development time for others.

D. Conclusions

It is possible to effectively nudge students frequently throughout the term if such a system is automatically connected to Outlook via Excel.
It is hoped that others will adapt this system for use in their courses to improve student engagement through nudges and report their findings. Topics for future research include but are not limited to the content of nudges, the tipping points selected, and the frequency of nudges.
Those not using MS Office will have difficulty implementing this system. While the code has been made as self-explanatory as possible some familiarity with Excel functions and VBA programming is needed to adapt this system.
A more in depth study correlating specific student comments to specific messages was not possible due to IRB concerns. The same is true regarding performance improvement as the result of these nudges.

A movie showing customization is available on markapalmertechonline.com.

The following web page has the code for all macros in Word and a sample Excel Workbook. These are available under the Attribution-NonCommercial-ShareAlike 4.0 International Creative Commons License

REFERENCES

[1] Zoë Cohen, Becky Supiano: How One Email From You Could Help Students Succeed; The Chronicle of Higher Education – Teaching Newsletter, 8/9/2018

[2] Colleen Flaherty: My Professor Cares; Inside Higher Ed, 1/14/2019

[3] Zoë Cohen: Small Changes, Large Rewards: How Individualized Emails Increase Classroom Performance, 7/17/2018

[4] Susan Ohrlabo: 3 Ways to Assess and Build Student Resiliency; Academic Impressions, 12/15/2014

The post Improving Student Success by Being Automatically Personal appeared first on ASEE Computers in Education Journal.

The Effect of An Automatic Feedback System on Students’ Comments to Improve their Performance

Francis Igot — Mon, 06 Jan 2020 16:17:22 +0000

Shaymaa E. Sorour

Dept. of Educational Technology
Faculty of Specific Education Kafrelsheikh University
Egypt
shaymaasorour@gmail.com

Hanan E. Abdelkader

Dept. of Computer Teacher Preparation
Faculty of Specific Education
Mansoura University
Egypt
h_elrefaey@yahoo.com

Abstract—This research focuses on understanding student performance by giving automatic feedback after writing freestyle comment data in each lesson. Writing comments express students’ learning activities, tendencies, attitudes, and situations involved with the lesson. Random Forest and Support Vector Machine were applied to analyze the students’ prediction results. Also, a Majority Vote (MV) method is employed in consecutive lessons during the semester to the predicted results. The proposed system tracks student’s learning activities and attitudes with different courses and provides valuable feedback to improve educational performance.

Keywords—Automatic feedback, Comment data, SVM, RF, MV

I. INTRODUCTION

Improving the performance of students, determining their actual progress, and enhancing their learning process can be extremely valuable in the educational environment. To achieve a high level of student performance, we have to find ways to measure their current progress and predict the results of the learning process at the earliest stages. It can be exceedingly vital to control and handle the difficulties from the beginning [1,2] Academic analytics deserves proper attention because it supports educational institutions in enhancing student accomplishment [3]. The prediction of students’ performance becomes possible because of the massive amount of educational processes data [4]. Students’ performance is the most critical goal for higher educational institutions. Where the university’s goal quality can be measured based upon its scores on academic achievements, that is considered the essential criteria. The previous literature stated that many define student performance by gains made by measuring their learning estimation [3]. Although, many studies consider students’ success measurement at graduation, where they are evaluating the performance of students using the final grades. Final grades contain the structure of courses, reports, activities, and final exam marks. The evaluation assists in the maintenance of the effectiveness and performance of the students’ learning processes. Also, through analyzing students’ performance, strategic programs can be perfectly designed over their studies time [5].

Evaluating students is the most effective method that precedes teacher evaluation at educational institutions. Two types of questions to introduce to the students: the fixed-ended questions and open-ended questions. [6] The fixed-ended questions do not allow the respondent to provide unique or unanticipated responses, but the students must choose from specified options. Open-ended questions allow students to give any answer they want without forcing them to select from concrete options. So open-ended questions give freedom in response and are uncommitted to match the standard answer categories. Besides, students’ answers, “freestyle comments” can take many expressions and provide valuable information [7]. Students’ comments reflect the complication of learning strategies, and the reaction to teaching environments provides additional beneficial information about essential educational issues and suggests excellent views to develop them [8-10]. Students’ comments analysis can provide useful ideas for teaching aspects and courses that can benefit students [11,12].

Currently, there are many proposed techniques for extracting useful information from student comments. Text mining (TM) is one of the most common methods to analyze comments and had been widely applied in educational environments. The text mining performs many processes such as it handles unstructured input, extracts significant numeric indices from the text, and prepares the information contained in the text to access data mining algorithms such as statistical and machine learning. Data can be extracted from the document’s words, the words can be analyzed and determine the similarities between documents and words [13,14].

In the educational context, feedback achieves the critical role of limiting the gap between students’ actual level and targeted findings [15]. Immediate and useful feedback can assist students in realizing and correctingmistakes, encourage them to obtain knowledge, and increase their motivation towards learning and achievement [16]. Automatic feedback has an essential role in informing the students of and providing the mechanism to correct their mistakes. It assists in modifying the learner’s behavior by evaluating its results. Automatic feedback also plays a severe role in the self-learning process; it supports and motivates students, contributes to increasing scientific learning efficiency, improves learning quality and speed [17,18]. Also, automatic feedback presents opportunities for students to enhance their performance by quick actions based on writing freestyle comments data from the proposed system.

In this paper, we built a system to collect comment data, provide automatic feedback to each student, and examine more applicable students’ prediction performance development models. There are particular student features related to the student success rate. Detecting the relationships between each student’s task and their accurate score is one of the efficient applications of student’s performance. Besides, automatic feedback could assist in enhancing students’ learning process through the semester [19, 20]. For more contributions to understanding students individually in the class, this research introduces an analysis of student comments to grasp students’ status and attitudes. Also by predicting their final grades and providing automatic feedback for each student to follow the anticipated results in the consecutive lesson.

II. RELATED WORK

In this section, the studies that present the automatic feedback were addressed in more detail as an essential tool to evaluate and guide the student for more progress at their educational issues. Besides, the importance of detecting students’ performance at the earlier stage and predict their performance using data mining and text mining techniques were displayed. Zenobia et. al [21] aimed at helping nurse educators /academics realize the student’s expectations and views displaying their feedback to educators about teaching performance and quality of the subject. There were four essential subjects about collecting feedback at more than one time; adjusting the questions being asked to students for collecting feedback; determining the collecting feedback methods efficiency and presenting the next step after student feedback. They approved the importance of examining the issue from the student opinion exploring the collecting feedback timing and channels and displaying the preferred uses of student feedback. Further, [22] provided automated feedback depend on the analysis of text mining to decrease the plagiaristic behavior of learners’ online assignments. Analysis of document similarity had performed twice at the middle and end of the semester about the computer science concepts course. The results of the analyses detected a statistically significant difference in the plagiarized posts ratios and the ratios of students performing plagiaristic behavior before and after feedback. In addition, because of huge amounts of data that the E-learning systems generate, whose analysis became a terrible job, which makes it serious about utilizing computational analytical techniques. [23] proposed using knowledge discovery techniques for analyzing historical course grade data of students to predict the probability of students drop out of a course. The significant contribution was a tutoring plan that could be applied by the learning institutions (and others) to decrease the dropout rate in e-learning courses.[24] used data mining methods to study the undergraduate student’s performance where the data mining domain assists to mine educational data for improving the educational process quality. Two students’ performance aspects had been concentrated upon. Firstly, predicting academic achievement. Secondly, studying and combining typical progressions with prediction results. Low and high achieving students’ groups had been identified. The results proved that it is the potential to give immediate warning and support to low achieving students, give advice and chances to high performing students, by centering on a small number of courses indicates good or bad performance. [25] approved that feedback could be presented to learners depends on comparable solutions from the asset of stored examples. They present a framework to build metrics on sequential data and using machine-learning techniques to adapt to those metrics. The adaptable metric recovers the wrong versus correct learner attempts from sports training in a simulated data set, the underlying learner strategy classification in a real Java programming dataset. [26] detected the effect of applying blended learning (BL) on higher education student’s academic achievement. Statistical synthesis of studies comparing student performance in BL with traditional classroom instruction was performed. The finding emphasizes that BL is significantly associated with better learning performance than with traditional classrooms. Therefore, to improve student performance, there was discussion around the implications and returns for future research. [10] indicated that comprehensive research had been executed on student ratings on closed-ended questionnaires, research has been executed on student ratings on closed-ended questionnaires, but there is a lack of research that has examined the responses of students to open-ended questions. They inspected the students’ written comments in 198 classes, based on their frequency, content, orientation, and consistency with quantitative ratings on closed-ended items. These comments were directed to be positive, not negative, and tended to be general rather than specific. The new automated scoring techniques applications’, like natural language processing and machine learning, enable automated feedback on students’ short written responses. Although numerous researches approved the automated feedback in the computer-mediated educational environments [27]. Feedback Properties in various studies treat the feedback timing, and according to timing, feedback can be immediate or delayed. In the educational context, immediate feedback was commonly more efficient than delayed feedback [15,28]. Immediate feedback can discover and evaluate mistakes before students fully understand academic conceptions [29]. The immediate feedback utility would be much more in computer-based educational climate feedback that can be reached to each student relying on their performance [15,30].

Previous studies approved that feedback is valuable in supporting learning, and it is not easy to provide students’ automatic feedback. Also, writing comments data that express student’s attitudes and knowledge has vital rules to improve student performance.

III. BACKGROUND

Giving feedback to students after each lesson has a vital role in improving their performance. The proposed system collected comment data for 15 lessons automatically by providing an online website, as shown in “Fig. 1”. The proposed system provided some instructions to improve students’ writing skills and extract significant assignments to supply common and comprehensive instructions in each lesson during the semester. “Fig. 1” shows the system interface after signing in. It presents 15 lessons and 15 feedbacks to write comments and provide feedback after writing comments.

Fig. 1 System interface that includes lessons and feedback

Fig. 2. Five questions and answers

Comments data were collected from the students’ answers on five questions, as shown in “Fig. 2” wherefore students could express their educational situation. Writing comment data reflect students’ knowledge, activities, attitudes, and tendencies for each lesson. Students think back, find out their problems to improve their learning, and the teacher displays some counsels that make students adequately depict their comments. Question (1): which are “Review the lesson.” Students illustrate their achievements of the previous lecture. Question (2): Each student writes the understanding or difficult points for the current lecture. For question (3): students describe what they detected and recognized information and the efforts they made for the lesson. In (Q4): Students express the cooperative activities sharing their friends for solving problems. Question (5): about the “Next plan.” Students indicate their next outline before the next lecture. Students can view and edit their comments before submitting them, as shown in “Fig. 3”.

Fig. 3. View comments to submit them

IV. SYSTEM OVERVIEW

“Fig. 4” shows the proposed system overview from collecting high-quality students’ comments data, predicting student performance, providing automatic feedback in order to improve students’ performance.

Fig. 4. System Flowchart

Students write their comments in the English language after each lesson, as shown in “Fig.” 1. 122 undergraduate students at the Faculty of Specific Education, Department of Educational Technology. Specialization of Computer Teacher Preparation, from Kafrelsheikh University, Egypt. Students” took Multimedia and Applied Statistics courses that consisted of 15 lessons. They registered the system and assigned in by their emails. They write comment data after each lesson where the proposed system provides editing property before submitting comment data. Students can edit their comments before submitting them. The system sent emails to all students that contain their comments.

A. Students’ Levels

Students’ marks were classified into five levels, as shown in Table 1, the students’ grades, many students, and students’ degrees. The evaluation process is considered the rate of attendance and the average mark of student’s exercises and reports for six assignments, which covered the subjects from lessons 1 to 4, lessons 1to 6, lessons from 1 to 8, lessons 1 to10, lessons, lessons 1 to 15.

Table 1. Number of Students’ Level

GRADE	NUMBER OF STUDENTS	DEGREE
A	70	100-85
B	20	84-75
C	20	74-70
D	8	69-50
E	4	49-0

Students’ number of level D and level E is smaller than other levels for predicting their grade, so comment data in levels D and E were combined in one level D.

B. Students’ Groups

In this research, we classified students into two groups:

Group 1 (comment data only): Students write their comments after each lesson
Group 2 (giving feedback): The system provided automatic feedback to each student after writing feedback in each lesson from 1 to 15.

C. Morphology Analysis

The proposed system analyzed students’ comments automatically utilizing the TF- IDF method with NLTK by extracting parts of speech of the words (e.g., noun-verb – adverb and adjective). TF: refers to a word frequency and expression in a text comment and IDF: indicates to the number of comments in the corpus divided by the number of comments that the word or expression appears in terms with higher. In addition, the system applied an Attribute Dictionary (AD), which express positive and negative expressions automatically from students’ comments. First, the system concentrated on verbs and negative auxiliary verbs; in English, did not is commonly used to display negative verbs for past tense, did not for present tense.

The system stems from the negative auxiliary verb. Then extracts the phrases before verbs till subjects. Furthermore, it extracts positive verbs and the whole phrases that contain higher frequencies of positive. A value of an attribute assigned to the extracted words for the student’s in each question to build an automatic AD; the value is one of Positive (Pos.), Missing (Mis.), and Negative (Neg.). Positive attitudes such as: (do the best, attractive, able to, manage to, can do). Negative attitudes such as: (cannot do, confused, afraid, unable to, frustrated). Words and expressions with high frequencies were extracted from all lessons that express students ‘attitudes toward the lesson. Then one of the three value (Pos, Mis, or Neg) were specified to predict student performance.

This research aim is to focus on verbs (negative/positive) with (5) questions from lessons 1 to lesson 15 to assign the attribute. Students’ comments were translated into positive or negative attributes. However, some of the students described their comments without writing the verbs. So, other extracted words were checked to allocate the correct attribute.

D. An Advisory System

To build an advisory system, automatic feedback was provided to each student from the first lesson to each student according to the positive and negative expressions. If students’ comments did not contain positive or negative expressions, the teacher could give feedback by himself, and the system saved all feedbacks to each question to use them again with other lessons. “Fig.5” shows how to give feedback to students.

Fig. 5 Add useful comments and save all feedback on the list.

Our system can distinguish between different cases, as shown in “Fig. 6” who write their comments and had feedback with yellow color. Who did not write their comments in red color? Who write their comments, and the system did not provide feedback with gray color; the system needs help from the teacher to add unique feedback?

In order to motivate students to write comment data that express their attitudes and the current situation, the system can select a useful comment, and all students can see this comment and who write it as shown in “Fig. 5”. Also, the system saves all feedback from all students. The teacher can select from them to add feedback, as shown in “Fig.5”.

Realizing how to use the previous feedback, our system calculated the feedback used from the previous lessons and the new feedback used in the current lesson, as shown in Fig. 7. From lesson 7, the system did not count any further feedback. It depends on the previous feedback.

Fig. 6. The system can distinguish between different students who write comments and have feedback

E. Prediction Models

In this study, two experimental groups were classified in each course. The first group students writing freestyle comments data, and the second group, the automatic feedback was given after each lesson. SVM and RF models are applied to predict student performance, and the Majority Vote method was applied to the attribute vectors from students’ comments to develop the prediction.

a. Support Vector Machine

SVM is a supervised learning model that aims to analyze and utilize data for classification and regression analysis, additional randomness to the model while growing many trees. It searches for the best feature among a random subset of features. In our research, the best results for all the lessons were obtained using 10 trees to compose RF for 15 lessons [31] SVM trains algorithm, creates a model that allocates different examples to any of the groups, forms it as a non-probabilistic binary linear classifier. New examples are then predicted and mapped into the similar. By comparing SVM with other machine learning techniques, as Artificial Neural Networks, SVM has a better implementation. So, SVM has achieved perfect performance in several predictive data mining applications such as text categorization, time series prediction, and pattern recognition, etc. [32].

b. Random Forest

RF algorithm is a classifier of reference. It forms the majority of current machine learning systems, and its success is based on creating the forest with several trees that accurate the prediction results. Random Forest adds [33].

V. EVALUATION METHODS

A10-fold cross-validation was applied to predict student performance from lesson 1 to 15. Let (L) a set of students. Students’ levels are {A; B; C; D}. Where TP refers to true positive (TN) refers to true negative (FP) refers to false positive, and (FN) refers to a false negative. To evaluate the proposed methods, students’ comments were measured by F- measure (F1); it is the mean of Precision (Pl) and Recall (Rl). The overall F1 result can be calculated by macro-average (F1Ma) [34].

A. Predict the Student’s in Consecutive Lessons

The goal of this part in two ways: First, to show the effect of writing comment data during the semester in two courses, which are different to grasp students’ characteristics in consecutive lessons. Second, to focus on the feedback roles in a range of lessons; to understand students’ learning situations and attitudes. The Majority Vote method is applied [35] for predicting students’ performance in consecutive lessons.

VI. RESULTS

The results were performed in two ways. First, to conduct the comparison of RF model with that of SVM model to predict student performance in two different courses. Second, to test the effect of automatic feedback method in consecutive lessons during the semester.

Fig. 8. Overall F1 (Ma) results for each

Fig. 9 . Overall F1 (Ma) results for each level

Fig. 10. F-measure (F1) from lessons 1 to s for Multimedia Course

Fig. 11. F-measure (F1) from lessons 1 to s for Applied Statistics Course

A. SVM and RF Prediction Results

Fig. 8 shows the F1Ma results of final student grades for the Multimedia course with the RF and the SVM models using comment data only (Group1) and after giving feedback to students (Group2). SVM model had the best prediction results for all lessons than the RF model. The average overall F1Ma results were 0.90 and 0.89 for SVM and RF for Group 1 and 0.95 and 0.92 after giving feedback to students (Group2). For the Applied Statistics course, the average overall F1Ma results were 0.89 and 0.87 for SVM and RF for Group 1, 0.94, and 0.92 for Group2.

B. The Characteristics of Students’ Grades Between the Two Groups

Tables 2, 3, and “Fig. 9” show the differences in students’ grades in two groups for the Multimedia course using RF and SVM models. SVM achieved the highest results. Also, there is no difference between grades (A, B, C, D) on prediction performance.

Table 2. Overall Prediction Results for Group 1 (Comment data only)

GRADE	RECALL	PRECISION	F-MEASURE
A	0.98	0.91	0.94
B	1.00	0.90	0.95
C	0.93	0.83	0.87
D	0.91	0.79	0.85

Table 3. Overall Recall, Precision and F-measure results for Group 2 (give feedback)

GRADE	RECALL	PRECISION	F-MEASURE
A	0.98	0.96	0.97
B	1.00	0.93	0.96
C	0.91	0.96	0.94
D	0.89	0.96	0.93

VII. CONCLUSION

This paper discussed a proposed system which connects with freestyle comment data and automatic feedback to understand students’ attitudes, activates, treat actual gap, realize the current situation, and increase motivation. The proposed system adds some features to collect high-quality comment data that present the current situations in all lessons with different courses. SVM and RF were used, and the prediction results (F-measure) were higher for the two groups mainly from the last 7 lessons. Also, Group2 (giving feedback) achieved the best results from all lessons for the two courses. To improve the prediction results in consecutive lessons, the Majority Vote method was applied to the two groups.

To provide feedback, positive and negative expressions were extracted by distinctive words for each question. Furthermore, other attributes from students’ comments were treated to handle more profound semantic terms and provide valuable feedback that enhances students’ performance. In the near future, we will try other methods that investigate such words in all courses that are clues to improve students” performance and overcome their educational problems.

REFERENCES

[1] Carlos V., et al (2017) Improving the expressiveness of black-box models for predicting student performance, Computers in Human Behavior, 72, 621-631.

[2] Berland, M., Baker, R. S., & Blikstein, P. (2014). Educational data mining and learning analytics: Applications to constructionist research. Technology, Knowledge and Learning,19,205-220.

[3] Usamah, b. M., Buniyamin, P. M. Arsad, & Kassim,R. (2013), An overview of using academic analytics to predict and improve students’ achievement: A proposed proactive intelligent intervention, in: Engineering Education (ICEED), IEEE 5th Conference on, IEEE, 126–130.

[4] Mohamed.,A. S.,Husaina,W., & Nuraini, A. R., (2015). A Review on Predicting Student’s Performance using Data Mining Techniques, The Third Information Systems International Conference, Procedia Computer Science,72,414 – 422.

[5] Ibrahim,Z., & Rusli,D. (2007), Predicting students’ academic performance: comparing artificial neural network, decision tree and linear regression, in: 21st Annual SAS Malaysia Forum, 5th September.

[6] Arnold, I. J. (2009). Do examinations influence student evaluations? International Journal of Educational Research,48(4), 215-224.

[7] Brock, B., Van Roy, K., & Mortelmans, D.,(2012). The student as a commentator: students’ comments in student evaluations of teaching, International Conference on Education and Educational Psychology (ICEEPSY 2012), Procedia – Social and Behavioral Sciences, 69, 1122 – 1133.

[8] Sorour, S. E., Mine, T., Goda, K., & Hirokawa, S. (2015). A predictive model to evaluate student performance. Journal of Information Processing (IPSJ), 23(2), 192-201.

[9] Johnson, B., & Christensen, L. B. (2014). Educational Research: Quantitative, Qualitative, and Mixed Approaches: Sage Publications.5th.Ed. Library of Congress Cataloging-in-Publication Data, United States of America.

[10] Alhija, F. N.-A., & Fresko, B. (2009). Student Evaluation of Instruction: What Can Be Learned from Students’ Written Comments? Studies in Educational Evaluation, 35(1), 37-44.

[11] Hodges, L. C., & Stanton, K. (2007). Translating Comments on Student Evaluations into the Language of Learning. Innovative Higher Education, 31(5), 279-286.

[12] Oliver, B., Tucker, B., & Pegden (2007). An investigation into student comment behaviors:Who comments, what do they say, and do anonymous students behave badly? Paper presented at the Australian Universities Quality Forum, Hobart.

[13] Agrawal,R., & Batra.,M (2013), A Detailed Study on Text Mining Techniques, International Journal of Soft Computing and Engineering (IJSCE), ISSN: 2231-2307, 2(6).

[14] Arpita G., & Anand S. (2015). Sentimental Analysis Techniques for Unstructured Data, International Journal of Advanced Computational Engineering and Networking, ISSN: 2320-2106, 3(9).

[15] Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research,78(1),153–189, https://doi.org/10.3102/0034654307313795.

[16] Epstein, M. L., Lazarus, A., Calvano, T., Matthews, K., Hendel, R., & Epstein, B. (2010). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52(2), 187–201.

[17] Cheng,G. (2017): The impact of online automated feedback on students’ reflective journal writing in an EFL course, The Internet and Higher Education, 34,18–27.

[18] Shermis, M. D., & Burstein, J. (2013). Handbook of automated essay evaluation: Current applications and future directions. New York: Routledge. Publisher: Routledge, Informa Ltd Registered in England and Wales. Registered Number: 1072954 Registered office: 5 Howick Place, London SW1P 1WG, UK

[19] Natek, S. k., & Zwilling, M. (2014). Student data mining solution–knowledge management system related to higher education institutions. Expert Systems with Applications,41,6400-6407.

[20] Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12-27.

[21] Zenobia, C.Y. Chana, Stanley, D.,J.,Robert J. M.,& Chiena,W.,T.(2017) A qualitative study on feedback provided by students in nurse education, Nurse Education Today, 55, 128–133.

[22] Akçapınar,G. (2015). How automated feedback through text mining changes plagiaristic behavior in online assignments, Computers & Education 87,123-130.

[23] Burgos,C., Campanario,M., Peña,D., JuanA. Lara, Lizcano,D., & Martínezv,M.,A.(2017). Data mining for modeling students’ performance: A tutoring action plan to prevent academic dropout, Computers and Electrical Engineering, 113, 1–16.

[24] Asif,R., Merceron,A., Ali,S.,A., & Haider,N.,G.(2017). Analyzing undergraduate students’ performance using educational data mining, Computers & Education, 113, 177-194

[25] Benjamin, P. n, Bassam, M.,& Barbara, H. (2016). Adaptive structure metrics for automated feedback provision in intelligent tutoring systems, Neurocomputing,192,3–13

[26] Hien M. , Zhu.C., & Nguyet A. D. (2017), The effect of blended learning on student performance at course-level in higher education: A meta-analysis, Studies in Educational Evaluation, 53 ,17–28

[27] Zhua , M.,Liua,O.,L., & Lee,H.(2020). The effect of automated feedback on revision behavior and learning gains in formative assessment of scientific argument writing, Computers & Education, 143, 112-126.

[28] Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of Asynchronous Learning Networks, 5(2), 1–17. https://doi.org/10.24059/olj.v5i2.1875.

[29] Dihoff, R. E., Brosvic, G. M., & Epstein, M. L. (2003). The role of feedback during academic testing: The delay retention effect revisited. Psychological Record, 53,533–548.

[30] Chen, X., Breslow, L., & DeBoer, J. (2018). Analyzing productive learning behaviors for students using immediate corrective feedback in a blended learning environment.

[31] Ghaddar,B.,&Naoum,S.,J.(2018). High dimensional data classification and feature selection using support vector machines, European Journal of Operational Research,265, 993–1004

[32] Panjaa,R., & Nikhil R. (2018) MS-SVM: Minimally Spanned Support Vector Machine, Applied Soft Computing, 64,356–365

[33] Abellán,J. Mantas,C., Castellano,J., & Moral,G.,S.(2018). Increasing diversity in random forest learning algorithm via imprecise probabilities, Expert Systems With Applications, 97,228–243.

[34] Yang, Y., & Liu, X. (1999). A re-examination of text categorization methods, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval,pp. 42-49.

[35] Luo, J., Sorour, S. E., Goda, K., & Mine, T. (2015). Correlation of grade prediction performance with characteristics of lesson subject, Proceedings of the IEEE 15th International Conference on Advanced Learning Technologies, 247-249. doi: 10.1109/ICALT.2015.24.

BIOGRAPHICAL INFORMATION

Dr. Shaymaa E. Sorour, PhD

Assistant Professor, Computer Science and Education, Department of Educational Technology, Faculty of Specific Education
Kafrelsheikh University
Egypt

She is an assistant professor of Computer Science and Education, Department of Educational Technology, Faculty of Specific Education, Kafrelsheikh University, Egypt. She is a Director of the Quality Assurance Unit – Faculty of Specific Education – Kafrelsheikh University. She received her PhD 2016, in Computer Science and Education, Department of Advanced Information Technology, Faculty of Information Science and Electrical Engineering, Kyushu University, Japan. Her Specialization are Computer Science – Artificial Intelligence – Machine Learning Algorithms. She received the best paper awarded in 5th IIAI International Congress on Advanced Applied Informatics, July 2016.

Dr. Hanan E. Abdelkader, PhD

Assistant Professor, Computer Science and Education, Department of Computer Teacher Preparation, Faculty of Specific Education
Mansoura University
Egypt

She is an assistant professor of Computer Science and Education, Department of Computer Teacher Preparation, Faculty of Specific Education, Mansoura University, Egypt. She received her PhD, in 2013 in of Computer Teacher Preparation Department , Faculty of Specific Education, Mansoura University. She is a Certified instructor at the Professional Academy for Teachers – Ministry of Education – Egypt – from 1/10/2017. Humanities Sector Coordinator at the Office of Vocational Guidance for Rehabilitation and Continuous Training headed by Vice President for Education and Student Affairs – from 01/29/2017 to 28/1/2018. Her Specialization are Computer Science – Artificial Intelligence – Web Mining.

The post The Effect of An Automatic Feedback System on Students’ Comments to Improve their Performance appeared first on ASEE Computers in Education Journal.

ASEE Computers in Education Journal

Exploring Literate Programming in Electrical Engineering Courses

Related ASEE Publications

Introduction

Background Literature

Literate Programming (LP)

Cognitive Load Theory

CodeChat: A Modern LP Approach

Research Overview

CASE 1: Microprocessors Course

Approach for Microprocessors

Data Collection for Microprocessors Case

Results for Microprocessors Case

Discussion of Microprocessors Case

CASE 2: Digital System Design Course

Approach for Digital System Design

Data Collection for Digital System Design Case

Results for Digital System Design Case

Discussion of Digital System Design Case

Conclusions

Analysis of Aircraft Actuator Failures within an Undergraduate Experiential Learning Laboratory

Related ASEE Publications

Introduction

Aircraft Dynamics under Actuator Failure Conditions

Basic Experimental Design Concepts

Simulation Tools

﻿Laboratory Objectives and Learning Outcomes

Laboratory Outline and Instructions

Academic Impact and Student Feedback

Conclusion

Simulation and Interactive Digital Tools to Support Teaching Engineering Manufacturing Processes Course

Related ASEE Publications

Introduction

Course Description

Course Implementation

In-class Simulation Projects and Exercises

Project #1 Tolerances Design for a linkage pivot

Project #2 Sand casting of a wolf head

Heating and Pouring

Solidification and Cooling (Chvorinov’s rule)

Online Quizzes

Students’ Micro-lectures (MLs)

Results And Discussion

In-class Simulation Projects and Exercises

Online Quizzes

Students’ Micro-lectures (MLs)

CONCLUSIONS

Promoting STEM to Middle School Girls through Coding and Fashion

Barbara L. Stewart

Carole Goodson

Susan L. Miertschin

Susan L. Schroeder

Misha Chakraborty

Marcella Norwood

I. INTRODUCTION

II. REVIEW OF LITERATURE

A. Demand for STEM Skills

B. Benefits and Effects of Camps

III. CAMP DESIGN AND DEVELOPMENT

IV. CAMP ASSESSMENT AND RESULTS

CONCLUSIONS

REFERENCES

BIOGRAPHICAL INFORMATION

Barbara L. Stewart

Carole Goodson

Susan L. Miertschin

Susan L. Schroeder

Misha Chakraborty

Marcella Norwood

Innovative VR-Based Research to Develop Intuitive Human Computer Interaction

Magesh, Chandramouli Computer

I. INTRODUCTION AND BACKGROUND

II. VR AS AN INSTRUCTIONAL TRAINING TOOL IN STEM & NON-STEM CONTEXTS

III. METHODOLOGY: CONCEPTS, HARDWARE & SOFTWARE

IV. RESULTS AND DISCUSSION

V. CONCLUSION

ACKNOWLEDGEMENTS

VI. LIMITATIONS

REFERENCES

A Pattern Recognition Framework for Embedded Systems

Laboratory Objectives and Learning Outcomes

Frank Vahid

Tony Givargis

Roman Lysecky