US20100242709A1 - Interactive game providing instruction in musical notation and in learning an instrument - Google Patents

Interactive game providing instruction in musical notation and in learning an instrument Download PDF

Info

Publication number
US20100242709A1
US20100242709A1 US12/789,271 US78927110A US2010242709A1 US 20100242709 A1 US20100242709 A1 US 20100242709A1 US 78927110 A US78927110 A US 78927110A US 2010242709 A1 US2010242709 A1 US 2010242709A1
Authority
US
United States
Prior art keywords
game
musical
keyboard
midi
challenge
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/789,271
Inventor
Hal Christopher Salter
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US12/789,271 priority Critical patent/US20100242709A1/en
Publication of US20100242709A1 publication Critical patent/US20100242709A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/06Electrically-operated educational appliances with both visual and audible presentation of the material to be studied
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00Teaching music
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00Teaching music
    • G09B15/02Boards or like means for providing an indication of notes
    • G09B15/023Electrically operated
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00Teaching music
    • G09B15/02Boards or like means for providing an indication of notes
    • G09B15/04Boards or like means for providing an indication of notes with sound emitters

Definitions

  • This application contains an Appendix comprising (1) CD-ROM containing specifications of computer programs used to carry out the invention disclosed herein.
  • the contents of the CD-ROM are described in more detail in paper Appendix A attached to this document.
  • the contents of the CD-ROM are incorporated into this disclosure by reference in their entirety as if they were set forth completely within the text of this application.
  • This present invention relates generally to an interactive game, and more particularly to a process for interpreting or compiling MIDI or other standard music files into an interactive game that instructs its used in musical notation while playing the game.
  • MIDI technology has been a stable source of piano and music sequencing programs for editing, printing, playing and composing music for almost twenty years.
  • some types of computer aided instruction, including piano instruction have been used in the home market for several years.
  • a major problem with these prior art learning systems is that they fail to take into account several important differences between learning to type and learning to play music.
  • the first and most obvious difference is that musical notes cannot merely be played in a correct sequence, but must be played in a rhythmic pattern in relation to the notes before and after it.
  • the correct timing of the notes is one of the most difficult aspects to teach in music, by any technique; and up to now few applications have been able to adequately address this challenge, and then only in isolation from other concepts.
  • the second challenge is that most programs start with musical notation to teach a song to the user. Reading and understanding musical notation is a bit like reading Beowolf in the Old English, and it's roots are just as old.
  • the five line staff originally represented a monk's hand in the Middle Ages, and has been adapted to deal with all the musical innovations since.
  • U.S. Pat. No. 4,416,182 to Wise, et al. discloses a keyboard teaching device for the self-instruction of a student of keyboard musical instruments. They system enables the student to correlate the positions of the keys on a musical instrument keyboard with the positions of the notes on a musical scale.
  • a keyboard having a plurality of keys corresponding to the notes of a musical scale generate a first set of control signals in an initialization or set-up mode and key-note correlation signals in an instruction or game mode.
  • There is at least one storage element storing a predetermined combination of logical signals providing a source for a pseudo random sequence of one or more notes over a predetermined range in the game mode.
  • the system generates a second set of control signals in response to the actuation of one or more keys in either game mode and a mechanism for generating audio tones and displaying video images in accordance with the first and second sets of control signals enables a student to visually and audibly check his or her selection of one or more keys.
  • U.S. Pat. No. 5,183,398 to Monte, et al. discloses an apparatus and method for instruction of a student which includes interactive guidance of the student through a series of lesson frames.
  • This disclosure provides the student with a keyboard having a plurality of keys corresponding to the notes of a musical scale and generates a key relation signal in response to each depressed or released key.
  • a video display and an audio tone generator associated therewith enables the student to respond to the visually displayed images and audio tones by selecting one or more of the keys on the keyboard.
  • the student is presented with a lesson frame representing an instructional activity requiring a response by the student on the keyboard.
  • the student keyboard response is compared with a performance standard and an absolute performance evaluation result in generated.
  • the absolute performance evaluation result is compared with an acceptable achievement level for the particular instructional activity and a next frame selection signal is generated.
  • a next frame is selected for presentation to the student based upon the next frame selection signal.
  • U.S. Pat. No. 4,997,374 to Simone discloses a teaching device that includes a changeable two channel prerecorded program source, and a console unit including a work booklet.
  • the first channel of the program source includes an audio program comprising a series of spoken words which are audibly reproduced by the console unit and the second channel includes a series of control signals which are operative for actuating lights adjacent prespecified words in the work booklet.
  • the operation of the console unit is coordinated with the audio program so that lights are actuated adjacent to the words in the work booklet as the same words are audibly reproduced by a console unit.
  • One embodiment of the device further includes a plurality of depressible user response buttons on the console unit for indicating responses to questions presented in the audio program.
  • U.S. Pat. No. 4,781,099 to Koike discloses a musical quiz apparatus that presents a question chord in sound and a trainee answers by depressing the keys of the chord constituting notes on the keyboard.
  • the apparatus is capable of generating a plurality of different chord data respectively representing chords, and generates a question chord datum one at a time more or less randomly selected from among those different chord data and produces sounds of notes which constitutes a chord designated by the question chord data.
  • points are added up and a next question chord is presented.
  • U.S. Pat. No. 5,392,682 to McCartney-Hoy discloses a computerized musical keyboard and a method for using same to play or to learn to play a piano.
  • the computerized musical keyboard includes a piano keyboard connected to a computer.
  • the computer is programmed to select from a music module a piece of music to be played on the piano and to generate a signal indicating the proper keys to be played, the correct sequence in which the keys are to be played, and the hand and finger to be used in striking each key, in order to play on the piano the piece of music selected.
  • U.S. Pat. No. 5,107,743 to Decker discloses a piano teaching aid having a panel designed to fit over the keys on an existing keyboard so that lights mounted on a panel having more than one color or shape may be located directly above the piano keys to be played. The lights have more than one color or shape in order to distinguish the hand which the user will use to play the piano.
  • the panel also includes a finder window which displays an alphanumeric code which corresponds to a like code appearing next to the score of music to be played.
  • a foot pedal advancing mechanism is used whereby the user can control the speed which the lights display the keys to be struck, using the Foot Pedal to advance one action.
  • the display also can show the music to be played at a tempo set by the user automatically changing from action to action without using the pedal.
  • the panel articulates so that it may be stretched in one or more places so that it can fit over various dimensions of keyboards without interfering with keys to be played.
  • U.S. Pat. No. 4,331,062 to Rogers discloses an apparatus for visually displaying music notes on a note display panel mounted on an electronic piano with a support arm.
  • the electronic piano has a keyboard electrically coupled to an electronic circuit operable to produce an audio output in accordance with the depression of one or more keys.
  • the arm is rotatable in a mount attached to the piano for movement about a first upright axis.
  • a first motion limiting unit attached to the mount and arm limits the rotation of the arm about the piano.
  • a second motion limiting unit attached to the panel and this arm limits rotation of the panel about an upright axis relative to the arm.
  • a modified structure has the arm fixed to the piano.
  • the panel has grand staff indicia coordinated with vertically disposed first light mechanisms corresponding to chord note information with a second light mechanism diagonally corresponding to ascending note scale information.
  • a keyboard representation is located below the staff indicia.
  • a third light mechanism associated with the piano key indicia of the keyboard representation is coordinated with the second light mechanism to provide visual information as to the keys depressed on the electronic piano.
  • An electric circuit having on-off switches electrically couples the electronic piano with the first, second, and third lights so that the lights can be selectively operated.
  • the electronic circuit has a switch assembly having a plurality of key signature switches operable to coordinate the first, second, and third lights with the piano keyboard in accordance with the key signature of the music that is played.
  • U.S. Pat. No. 4,366,741 to Titus discloses an electronic piano having a keyboard and an electronic piano circuit connected to a micro-processor used to control a CRT device to provide a video note display concurrently with the depression of one or more keys.
  • a keyboard representation located adjacent the screen of the CRT device is associated with lights used to indicate the key or keys that are played.
  • Manually operated controls cooperate with the micro-processor to allow the back clearing of the screen one note at a time, remove all the notes, retain all the notes, indicate sharp or flat mode of each note, and indicate the duration that a key is depressed by elongating the note on the screen.
  • a metronome unit is used with the micro-processor to provide a visual beat marker on the screen that sequentially moves across the screen.
  • a movable frame connects the CRT device to the piano.
  • U.S. Pat. No. 5,864,868 to Contois discloses a computer system and method for controlling a media playing device.
  • the system provides a user interface for allowing a user access to media pieces stored in a media database.
  • the interface is also for controlling a media playing device, like a player piano or movie playing video device, that is coupled to the computer to play the accessed or selected piece of media.
  • a media playing device like a player piano or movie playing video device
  • there is a computer interface that allows a user to display only music that relates to a selected category, like jazz or classical music.
  • Another embodiment allows the user to direct the media playing device to automatically play selected music pieces that are related to a selected music category.
  • Another embodiment allows a user to direct the media playing device to automatically play selected music pieces that are related to the selected music composer or artist.
  • U.S. Pat. No. 6,204,441 to Asahi et al. discloses techniques for displaying musical information and particularly for visually displaying musical notes, beats and tempos using personal computers or game devices that run musical software programs.
  • the disclosure teaches the use of different colors and different brightnesses to distinguish certain types of musical notation. It has a display screen which shows both base and treble clefs as well as the keyboard and timing indication.
  • U.S. Pat. No. 6,388,181 to Moe shows computer graphics animation, used with a live video interactive method for playing keyboard music while the user guides his fingers to the keys targeted by the animation with each key to be struck within one beat of time is designated by a colored “sprite”.
  • U.S. Pat. No. 6,066,791 to Renard et al. shows a system for instructing the playing of a musical instrument, displaying an image on the display device, and instructing the student to focus on the image while preferably using a musical instrument to play the notes on the staff.
  • U.S. Pat. No. 6,337,433 to Nishimoto et al. shows an electronic musical instrument having performance guidance function, performance guidance method, and storage medium storing a program therefore with a plurality of display devices arranged in association with the performance operating elements, respectively, each comprising a pair of display elements corresponding to left and right hands of the player, respectively.
  • U.S. Pat. No. 6,284,961 to Kimmel, Jr. shows a system of musical notes with the notes being associated with a color and utilizes stickers for application to the keys of the musical instrument to correspond to the colors of the note which it plays.
  • the invention is related to a hand-eye coordination game which has the purpose of teaching one how to read musical notation and how to play a musical instrument such as the piano.
  • the invention simplifies musical notation into a grid, with pitch moving left (lower) to right (higher), parallel with the piano keyboard layout, and the dimension of time moving vertically.
  • the player in the beginning phases of the game, the player will see game objects, representing notes of a song, rising from near the bottom of the screen toward the virtual piano keyboard, and their vertical relationship to each other shall be a representation of relative musical time.
  • game objects representing notes of a song
  • their vertical relationship to each other shall be a representation of relative musical time.
  • the corresponding piano key along that path is visually obvious, and the objective of the player is to hit the corresponding key on a musical keyboard at precisely the time the game object is located within a predefined hit window vis a vis the virtual keyboard. If the player strikes the correct key at the correct time, audio and visual feedback will reward the player.
  • the player's main task in the invention is to hit the corresponding piano key for as of the many moving objects as possible at the right time, repeating the challenge until they have reached a certain percentage of correctness and move up to the next level of complexity, and in doing so, subconsciously learn to play a song, and in the later levels, to recognize and read the sheet music.
  • FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the invention.
  • FIG. 2 is a diagram showing a virtual keyboard, game object trajectories and an exemplary relationship of those to a treble clef of a standard musical notation.
  • FIG. 3 shows a block diagram of exemplary hardware and software modules for carrying out the invention.
  • FIG. 4 illustrates the format of a MIDI file.
  • FIG. 5 illustrates the format of a header chunk of a MIDI file.
  • FIG. 6 illustrates the format of a track chunk of a MIDI file.
  • FIG. 7 illustrates the format of META events from a MIDI file.
  • FIG. 8 is an example of a portion of a MTrk chunk of MIDI data together with exemplary semantics.
  • FIG. 9 illustrates an exemplary flow of data illustrating MIDI messages for the sequential playing of three notes.
  • FIG. 10 is a flow chart of an exemplary process for interpreting the MIDI data stream of FIG. 5 for generating game objects for display.
  • FIG. 11 is a block diagram showing an exemplary arrangement of game modules in accordance with one aspect of the invention.
  • FIG. 12 is a block diagram of an exemplary computer that can be used to implement various aspects of the invention.
  • FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the invention.
  • FIG. 1 illustrates a computer 100 comprising a processing element (CPU) and a display.
  • CPU processing element
  • a typical implementation would be a personal computer commercially available from a number of sources. Details of the internals of computer 100 and its use in a network environment in FIG. 12 .
  • a piano keyboard, 150 preferably having the capability of generating controlling data in accordance with the MIDI standard is connected to the computer through the user standard MIDI interface.
  • FIG. 2 is a diagram showing an exemplary virtual piano keyboard, game object trajectories and an exemplary relationship of those to a treble clef of standard musical notation.
  • a virtual keyboard is an object that is shown on the screen of the computer's display as part of a graphical user interface.
  • the virtual keyboard will comprise a plurality of individual keys 200 i corresponding to the white keys of a piano and a plurality of keys 200 j corresponding to the black keys of the piano.
  • the virtual keyboard represents the standard arrangement of keys on a keyboard such as a piano.
  • a trajectory 210 i corresponding, in this example, to only the white keys of the piano keyboard.
  • One of the white keys and an associated trajectory correspond to each of the notes of an octave of musical notation, in this case, starting with middle C on the left and progressing up the musical scale until reaching the C above middle C. Trajectories shown in bold correspond to those note found on a line of the treble clef in this example.
  • game objects are introduced at a location opposite the virtual keyboard along the trajectories 210 i .
  • Game objects then progress in animated fashion along the trajectory from near the bottom to the point where the game object encounters the key of the virtual keyboard 200 i .
  • One object of the interactive game is for a student to press the key of a (preferably MIDI) keyboard substantially at the same instants that the game objects intersects they key of the virtual keyboard on the graphical user interface.
  • a (preferably MIDI) keyboard substantially at the same instants that the game objects intersects they key of the virtual keyboard on the graphical user interface.
  • the user develops a certain facility for mapping the visual expression of the notes from a musical file which drive the generation of the game objects into actual fingering positions on a real keyboard which interfaces with a virtual keyboard.
  • the game detects when the user presses the correct key at the proper moment.
  • FIG. 3 shows a block diagram of exemplary hardware and software modules for carrying out various aspects of the invention.
  • this particular embodiment of the invention is described with respect to the use of MIDI files and music formatted in accordance with a MIDI standard, other formats for music are well known and can be utilized in alternative embodiments of the invention.
  • one or more files 300 formatted in a MIDI format are stored in, for example, a library, or downloaded in real time and fed to interpreter 310 where the MDI data file is interpreted and translated in to commands which drive the game object generation module 320 which causes the display of game objects on the graphical user interface on the display screen and provides audio output 370 to the speakers of the computer.
  • a graphics engine 350 and an audio engine 360 simplify the translation of game objects into audio and visual components driving the visual display and the audio output.
  • the game object generator has access to a library of objects 330 and to a set of game control parameters 340 . Each of these modules are described in more detail hereinafter.
  • FIG. 4 illustrates the format of a MIDI file.
  • the MIDI file format is utilized for purposes of illustrating the invention, but the invention is not limited hereto.
  • An example of another file suitable for carrying out the invention would be that of MPEG-4.
  • MIDI files are structured into chunks. Each chunk consists of a 4-byte chunk type, a 4-byte indication of length indicating of the length of the bytes contained in the data field.
  • chunks There are two types of chunks, namely header chunks (which have a chunk type of “MThd” ( 410 )) and track chunks which have a chunk type of “MTrk” ( 420 ).
  • a MIDI file consists of a single header chunk followed by one or more track chunks.
  • FIG. 5 illustrates the format of a header chunk of a MIDI file.
  • the header chunk comprises three fields, namely, chunk type, length and data.
  • the chunk type is MThd.
  • the length field contains the length in bytes of the chunk data part.
  • the “format” portion of the data field includes the MIDI file format which can be only formats 0 , 1 and 2 .
  • the “tracks” portion of the data field as a binary number indicating the number of track chunks contained in the MIDI file.
  • the “division” portion of the data field of the header chunk defines the default unit of “delta-time” for the MIDI file.
  • the remaining 15 bits indicate the number of “ticks per quarter note” to be utilized in representing and reproducing the music. If the most significant bit is a logical 1, then there are two components indicated by the remaining 15 bits. Bits 8 - 14 would indicate the number of frames per second (indicated as a negative number) and the least significant 8 bits represents the number of ticks per SMTPE frame.
  • MIDI files come in three variations. Format 0 contains a single track. Format 1 contains one or more tracks which are all played simultaneously. Format 2 contains one or more independent tracks which can be or are played independently of the others.
  • FIG. 6 represents the format of a track chunk of a MIDI file. It comprises a chunk type which, by definition is a MTrk type and a length field which indicates the length of the data portion in the track chunk.
  • the data portion of a track chunk comprises two elements. The first is a “delta_time” portion and “event” portion. delta_time is the number of “ticks” from the previous event and is represented as a variable length quantity.
  • a MIDI event has any MIDI channel message. These include channel voice messages and channel mode messages. Messages other than MIDI channel messages to be included in a MIDI file can utilize the SYSEX event.
  • META events are used for things like track-names, lyrics and que points, which don't result in MIDI messages being sent, but are still useful components of a MIDI file.
  • MIDI events have the general form shown in FIG. 7 .
  • MIDI events are preceded with the hexidecimal notation FF followed by a type field, a length field and a data field.
  • the type field is a single byte specifying a type of META event.
  • the length field contains a number of bytes of data following that field.
  • the data field includes 0 or more bytes of data.
  • META events have been defined in the standard that make implementation of the invention easier. These include the following:
  • a “sequence number” is an optional event which must occur only at the start of a track before any non-0 delta_time.
  • a text event is utilized for annotating a track with arbitrary text.
  • a Copyright notice event can be utilized where a Copyright notice is represented in ACSI text. It should be the first event on the first track of a MIDI file.
  • a sequence/track name provides the name of a sequence or a track in the file.
  • An instrument name provides the description of the instruments used on the track.
  • a lyric event provides the lyrics for a song. Normally, each syllable will have its own lyric-event, which occurs at the time the lyric is to be sung.
  • a marker event marks the significant point in the sequence such as the beginning of a verse.
  • a que point is utilized to include queues for events happening on-stage, such as “curtain rises”, “exit”, and the like.
  • An end of track event must be utilized to give the track a clearly defined length. This is essential if the track is looped or concatenated with another track.
  • a set tempo event sets the tempo in microseconds per quarter note. This means a change in the unit-length of the delta_time tick. The default tempo is a 120 beats per minute.
  • a SMTPE offset event specifies the SMTPE time at which the track is to start.
  • a time signature representing the standard time signature for a piece of music, such as 3/4 or 6/8 or 2/2.
  • a key signature event can specify the number of sharps or flats and a major or minor flag.
  • a sequence-specific META event allows a manufacture to incorporate sequencer specific directives into a MIDI file.
  • FIG. 7 illustrates the format of META events from a MIDI file.
  • FIG. 8 is an example of a portion of an MTrk chunk of MIDI data together with exemplary semantics.
  • an MTrk chunk is introduced with an MTrk identifier 800 followed by a length field 810 .
  • the plurality of ordered pairs of delta_time and event fields 820 - 1 through 820 - j then follow which represent the individual MIDI messages associated with the MTrk chunk.
  • the MIDI protocol consists of messages which are designed to allow synthesizers and sequencers to communicate what-sound-to-play information.
  • a typical MIDI message comprises three components. The first component begins with a hexidecimal 9 is followed by an identification of one of sixteen MIDI channels having a value of O-F.
  • the second component is a two byte sequence representing the key on the device that has been pressed. This corresponds to the notes of a keyboard that has been pressed in the case of a piano-type keyboard. The two byte value ranges from 00-7F.
  • the third component of a MIDI message is the velocity component which specifies the velocity with which the key was pressed or released. It, too, ranges in value between 00 and 7F.
  • the keyboard would send a “note-on” message comprising: 90 3C 40.
  • the corresponding “note-off” message would comprise: 80 3C 33.
  • the key was released more slowly than it was pressed as indicated by the velocity indication 33 in the release message (compared with the attack velocity 40 ).
  • MIDI messages may include a program (instrument) change, a pitch blend message, a control change message (e.g. pedal/switch foot change of state) and timing clock message.
  • MIDI messages are all one way. There are no acknowledgement messages sent from the receiver back to the transmitter. If a MIDI device does not know what to do with a message, it will ignore it.
  • MIDI messages which are specific to a MIDI channel are referred to as channel messages.
  • MIDI messages which affect the entire MIDI system or an entire MIDI device are known as system messages.
  • Channel and system messages are further divided into several classes.
  • the channel voice messages are messages which start, alter or stop a sound or sounds being played.
  • Channel mode messages effect the entire channel.
  • System realtime messages are those used by sequencers to regulate and synchronize timing. They do not contain data bytes.
  • System common messages include messages such as song position pointer, song select and the like.
  • System exclusive messages are generally used for device specific extensions to a MIDI protocol.
  • MIDI messages are sent and interpreted in realtime, it is desirable to reduce the volume of data that must be sent.
  • note-on and note-off messages it is quite common for several notes to be turned off and on more or less at the same time. In such cases, it is possible to send a single status command, such as note-on followed by a plurality of “note identifier, velocity” pairs without repeating the same status byte for each note that is turned on or off. This reduction in the transmission of status bytes is known as “running status” within MIDI messages. Only the data bytes that change are sent.
  • FIG. 9 illustrates an exemplary flow of data illustrating MIDI messages for the sequential playing of three notes.
  • two MIDI messages are shown per line. Each is introduced with a delta_time field.
  • the six messages shown in FIG. 9 are preceded with delta_time increments of 00, 08, 10, 18, 20 and 28.
  • the delta_time increments are followed by a MIDI command sequence.
  • the sequence 92 indicates the situation where a note is to be turned on. Following each 92 message is a message with a command field of 82 which indicates that the note is to be turned off. Following each command is a note identifier. Which in the case of the first line indicates the note C is to be turned on and off. In the case of the second and third lines, the notes identified are the note E and the note G, respectively.
  • the last component of each message is a velocity component. In the first line, the velocity 44 is a measure of how fast the key is pressed downwardly (e.g. attack velocity). In the following message, where the key is being released, the velocity 40 indicates that the key is released with a velocity 40 .
  • FIG. 10 is a flow chart of an exemplary process for interpreting the MIDI data stream of FIG. 5 as part of generating game objects.
  • a MIDI message is received representing a note on object of the form ⁇ delta_time> ⁇ Command> ⁇ Note> ⁇ Velocity> ( 1000 ) optionally followed by a MIDI message representing a note-off property of similar format ( 1010 ).
  • ⁇ delta_time> less traversal time, an object is launched on ⁇ Note> trajectory with optional length property of ⁇ delta_time (off)>- ⁇ delta_time (on)> ticks ( 1020 ).
  • steps 1000 - 1020 represent an interpretation of the incoming MIDI messages followed by the generation of a game object. Messages are received and interpreted on an ongoing basis in accordance with the steps 1000 - 1020 .
  • a window is defined within ⁇ units of time of impact with the virtual keyboard ( 1030 ).
  • the object is moved along the trajectory towards the virtual keyboard with each N ticks of the MIDI clock ( 1040 ). If MIDI keyboard input is received during an open window and if the value of the MIDI keyboard input equals the MIDI note value of the object about to impact the virtual keyboard, video and audio reward presentation (routines) are activated ( 1060 ). Otherwise, a video and audio sequence representing a failure can be activated. This process is repeated for each tick of the MIDI clock in an iterative fashion as indicated by the arrow going back to the top of step 1040 .
  • item. 330 represents a library of objects which allow the game to be modified by the substitution of various types of objects for corresponding different flavors of the game. Examples of the types of objects utilized are given hereinafter. Also shown in FIG. 3 is a set of parameters 340 which allow a user to customize the performance of the game or the behavior of the game in certain instances.
  • the audio engine 360 is preferably based on Microsoft® DirectSound 8 application programming interface.
  • the audio engine generally has the ability to play short .WAV files through the PCM channels of a sound card.
  • the sound engine is implemented as a class with constructors, destructors, loading of a .WAV file and playing of a .WAV file.
  • the sound engine is used primarily for playing sound effects associated with menu buttons, clicks, switches and game play feedback sounds. However, the invention could also work with other software engines.
  • the graphics engine 350 works in conjunction with the library of objects 330 and the game object generation module 320 to produce graphical objects on the screen that constitute respective implementations of the game objects utilized to play the game.
  • the invention is capable of loading and playing three types of song files: MIDI, MIDI Karaoke, and .amm.
  • the later is an internal song format described more hereinafter.
  • Standard MIDI files were described above.
  • the MIDI Karaoke file is essentially a standard MIDI file with song lyrics built into one of the tracks according to a certain format.
  • the invention supports the .KAR file format which is a prevailing file format in the shareware/public domain markets.
  • An .amm file is very similar in structure to a standard type 1 MIDI file.
  • the .amm format file will also hold annotations, highlighting and fingerings.
  • Annotations are simple text notes that are stored on a song's timeline. These annotations can hold valuable information to a user and can be displayed during the game play along with other scrolling game objects. They may be stored as text in the internal file, but on the screen these are shown in the form as speech balloons with or without arrows pointing to other game objects.
  • Some notes in an .amm file can be highlighted. This is essentially a flag that tells the system that a note should be displayed with an enhanced video effect such as a halo effect around it. Fingerings are little clues to a user as to which finger should be used to play a note on a piano keyboard. Fingerings are assigned to individual notes and hold a number in a range from 1 to 5. During game play, game objects that have fingerings assigned will scroll along with the number placed on them.
  • FIG. 11 is a block diagram showing an exemplary arrangement of game modules in accordance with one aspect of the invention.
  • a splash screen 1100 is displayed. This is centered on the desktop and enables the user to click on icons or links that would take them to the sponsoring company's website where they can download new songs for the game. Also, there may be a link to HTML documentation for the game.
  • the Welcome Screen 1110 allows four options. First, it allows selection of a main menu which is described hereinafter. Second, it allows access to an instant play mode of operation, described in conjunction with item 1140 hereinafter, which assumes all previously set up configurations of the last song played in the game. A third option from the main menu takes one to the credits screen where game credits are revealed in a visual way. Finally, the main menu 1120 has an option to exit the game and return to the operating system.
  • main menu 1120 there are three options. If a quick play option is selected, the user goes directly to the quick play mode described in conjunction with 1140 where the user can freely load any song, select a number of options and commence the practice. No profile needs to be selected or created.
  • the career mode a user will create a profile and will face a number of challenges, predefined within a number of levels and in a number of different piano methods. These are discussed in detail hereinafter in conjunction with career mode 1150 .
  • a game set up 1130 may be selected.
  • game set up a number of game objects can be configured as described more hereinafter.
  • Instant play 1140 allows the user to play the game without the hassle of going through start up options. While previous starting option parameters are assumed, including the last song played if the game was initially installed and no songs have been played yet, a default song is assumed. Instant play has no menu, it is just a short cut to the game play.
  • FIG. 12 is a block diagram showing details of an exemplary computer that can be used to implement various aspects of the invention.
  • the description of the invention which follows is exemplary. However, it should be clearly understood that the present invention may be practiced without the specific details described herein. Well known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention. At least portions of the invention are intended to be implemented on or over a network such as the Internet. An example of such a network is also described in FIG. 12 .
  • FIG. 12 is a block diagram that illustrates a computer system 100 upon which an embodiment of the invention may be implemented.
  • Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 coupled with bus 102 for processing information.
  • Computer system 100 also includes a main memory 106 , such as a random access memory (RAM) or other dynamic storage device, coupled to bus 102 for storing information and instructions to be executed by processor 104 .
  • Main memory 106 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 104 .
  • Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing static information and instructions for processor 104 .
  • a storage device 110 such as a magnetic disk or optical disk, is provided and coupled to bus 102 for storing information and instructions.
  • Computer system 100 may be coupled via bus 102 to a display 112 , such as a cathode ray tube (CRT), for displaying information to a computer user.
  • a display 112 such as a cathode ray tube (CRT)
  • An input device 114 is coupled to bus 102 for communicating information and command selections to processor 104 .
  • cursor control 116 is Another type of user input device
  • cursor control 116 such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 104 and for controlling cursor movement on display 112 .
  • This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
  • Computer system 100 operates in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106 . Such instructions may be read into main memory 106 from another computer-readable medium, such as storage device 110 . Execution of the sequences of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
  • Non-volatile media includes, for example, optical or magnetic disks, such as storage device 110 .
  • Volatile media includes dynamic memory, such as main memory 106 .
  • Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 102 . Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
  • Computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
  • Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 104 for execution.
  • the instructions may initially be carried on a magnetic disk of a remote computer.
  • the remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem.
  • a modem local to computer system 100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal.
  • An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 102 .
  • Bus 102 carries the data to main memory 106 , from which processor 104 retrieves and executes the instructions.
  • the instructions received by main memory 106 may optionally be stored on storage device 110 either before or after execution by processor 104 .
  • Computer system 100 also includes a communication interface 118 coupled to bus 102 .
  • Communication interface 118 provides a two-way data communication coupling to a network link 120 that is connected to a local network 122 .
  • communication interface 118 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line.
  • ISDN integrated services digital network
  • communication interface 118 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN.
  • LAN local area network
  • Wireless links may also be implemented.
  • communication interface 118 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
  • Network link 120 typically provides data communication through one or more networks to other data devices.
  • network link 120 may provide a connection through local network 122 to a host computer 124 or to data equipment operated by an Internet Service Provider (ISP) 126 .
  • ISP 126 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 128 .
  • Internet 128 uses electrical, electromagnetic or optical signals that carry digital data streams.
  • the signals through the various networks and the signals on network link 120 and through communication interface 118 , which carry the digital data to and from computer system 100 are exemplary forms of carrier waves transporting the information.
  • Computer system 100 can send messages and receive data, including program code, through the network(s), network link 120 and communication interface 118 .
  • a server 130 might transmit a requested code for an application program through Internet 128 , ISP 126 , local network 122 and communication interface 118 .
  • the received code may be executed by processor 104 as it is received, and/or stored in storage device 110 , or other non-volatile storage for later execution. In this manner, computer system 100 may obtain application code in the form of a carrier wave.
  • Quick play allows playing the game without going through the challenges of the career Mode (section 0). It allows for loading song files and playing them freely. Within the quick play a player can select either Training or Dynamic mode.
  • Training Mode allows a user to practice the game on any song that can be loaded into the game.
  • Dynamic Mode is similar to a Training mode with the exception that there is a number of different dynamics that can be switched on and off. These dynamics pose an additional challenge.
  • Each dynamic has a number of variables that can be adjusted and will gradually change on a player as the game is progressing. The following is an exemplary list of dynamics:
  • Load Song is a part of the game that involves loading a song stored locally on the computer's hard drive. Users are able to browse for a specific song on their hard drive through a file search or simplified song browser.
  • the invention will not be capable of loading songs directly off a Website into the game, it will feature a link that will open a web browser (e.g. Internet Explorer, or Netscape) to a section of the web site where more songs can be downloaded from. Users are able to download songs manually from a website and then open them from within the game. The link is provided on the splash screen.
  • a web browser e.g. Internet Explorer, or Netscape
  • Levels have a predefined number of Challenges specified and available (section 0). One can apply any challenge from within the level to any song from within the level. The level is completed when the level requirements are met. The level requirements can be, for instance, to complete a minimum of 7 out of 10 songs, and apply a minimum of 3 out of 5 challenges for each song played. Once the level requirements are met, the next level is unlocked. Higher level scores can be achieved by doing more than a minimum requirement. Additional effort is rewarded with more points, and unlocking of additional graphic themes.
  • Profiles are required to play the game in the career mode. Profiles keep track of the overall game progress in the career mode separately for each game user.
  • One of the first things a user must do, before he can start playing in the career mode, is to create a profile. Users are able to select an icon from a pool of available icons and assign it to their name, similar to the way users on Windows XP operating system can before they can start using the system. Then, the game will prompt for their name, and what mode they would like to start playing the game in, i.e. beginner, Student, or Teacher. All the above information is saved onto the computer hard drive and kept there along with the entire game progress for that user.
  • Level selection screen Users are able to select a level from within a piano method. Higher levels are unlocked by completing previous levels.
  • the layout of the level selection screen are similar to piano method selection screen.
  • Each level selection on the list is displayed with a brief description, and an icon.
  • a top players list will also be displayed for each level highlighted for selection.
  • Statistics of the progress accomplished on a level will also be displayed once a level is highlighted on a list. Exemplary statistics will include: total time spent playing, and a number of challenges completed so far within a level.
  • Each level has a predefined minimum number of songs and challenges that need to be completed in order to advance to the next level. Also, each level has a reward in form of a theme that can be unlocked when a level is completed.
  • Songs are the last item in the hierarchy of Methods and Levels. A number of songs are available for each level. Songs that are available within the career mode are pre-configured, and for instance, track editing is not allowed. Playing these songs is on an as-is basis without the ability to alter the game environment, like it is possible in a Quick Play mode described in section 0.
  • Songs that are available within piano methods can be in any file format that is supported by the invention. Songs available from within piano methods are irreplaceable from the operating system. In other words, users will not be able to replace these songs with ones that are less difficult, in order to score better in the game.
  • the Game features a number of challenges for its users.
  • the challenges are designed to assist in training in a variety of techniques.
  • This invention will feature the following types of challenges:
  • the hit window gradually closes to a challenge creator's predefined size.
  • the starting size is a user definable parameter. Players need to stay within the accuracy parameter to continue playing.
  • the challenge is to stay within a predefined accuracy level hitting primary beats.
  • Primary beats are defined by the challenge creator. There is no penalty for hitting other (than primary beats) notes within the hit window.
  • Off-Beats Challenge requires to stay within a predefined accuracy level, hitting off-beats. There is no penalty for hitting other (than off-beats) notes within the hit window.
  • the challenge is to hold down the piano keys for the correct duration of notes. Players need to stay within the accuracy parameter to continue playing, say 90%.
  • the screen rotates from vertical to horizontal position. Players need to stay within the accuracy parameter to continue playing. As users master this challenge, they will learn the relationships between the keyboard and standard musical rotation.
  • a series of visibility challenges are introduced to induce the player to memorize the piece of music that he/she is paying. For instance, a cloud could enter the game field and cover parts of the screen, making it difficult for a user to see the game objects. Visibility challenges are further defined during the development.
  • Certain sections of a song will have different velocity parameters (forte, piano, etc.) that users need to stay within. Players need to stay within the accuracy parameter to continue playing. This challenge will only be available on touch-sensitive piano keyboards.
  • Custom piano methods involves creating an empty method, naming it, creating levels within, and adding songs to levels.
  • Custom piano methods do not include rewards, high-score lists or graphical themes. All challenges can be played on any song that is part of the custom piano method.
  • Custom piano methods should be savable in a manner so that they can be copied and shared among users of the invention.
  • Custom Piano methods are folders with files, and simple configuration files. All levels are unlocked and playable in custom piano methods. The idea of custom piano methods is to allow teachers for creating assignments for their students that are users of the invention.
  • Starting Options are options that can be configured before a user starts playing the game.
  • the number of available starting options depends on the game mode. In some modes some options are not available, and are grayed out, or left out. However, regardless of what mode the game was started in, in order to start playing, one will have to go through these options.
  • One exception to this rule is an Instant Play which omits configuring the Starting Options, and assumes all default values for playing. Preferred availability of Starting Options in different game modes is included in the CD-ROM Appendix.
  • This option specifies the orientation of the screen. Some users may prefer to start the game play in a Vertical mode, whereas some will prefer the Horizontal mode. It is also possible to start the game play in a mode where the screen will gradually start rotating from Vertical to Horizontal orientation.
  • Annotations are included in the .amm internal file format and involve displaying a kind of “post-it” text notes on the game screen as annotated game objects scroll.
  • fingerings are included in the .amm internal file format.
  • game objects if marked so, will display a number from 1 to 5 serving as a guide to a user as to what finger to use in order to target the game object correctly on the piano keyboard.
  • This switch enables or disables the ability to access a Freeze Menu from within the Game Play, as discussed hereinafter.
  • This option allows the user to select how many beats or measures are on the screen vertically during game play, allowing fast songs enough anticipation to prepare for the next note.
  • This option allows for selection between different types of game objects.
  • Long objects are those that can be drawn to represent the actual durations of musical notes, for instance: caterpillars.
  • short objects are those that only represent the starting point of a musical note, and do not represent the duration, i.e. Soccer Balls or Balloons.
  • Notation is selected, instead of game objects a user will see musical notation during the Game Play.
  • This option allows for selection between different types of musical notation. It is only available if Notation is selected as a game object.
  • Theme selection is a set of GUI elements that allow for selection of different game themes.
  • Themes include backgrounds and different game objects. Different backgrounds can be combined with different game objects. However, themes that were not yet unlocked in the career Mode (section 0) are not available for selection.
  • This switch turns on/off the color of the game objects as well as the color of the virtual piano keyboard on the screen. Basically, if toggled ON—all game objects and piano keys are displayed color-coded, otherwise, these objects are black & white.
  • This section allows for a visual selection of the tracks in a loaded song file. Once a song is loaded, a list of available tracks are displayed, and a user will have the ability to select those tracks that he/she wishes to play in the game. Once a track is selected by a user, the program will automatically find the range of musical notes for it, and mark it on the piano keyboard displayed on the screen (i.e. C2-B3). The range will always be calculated for all selected tracks. For instance, if two tracks are selected: track1 and track2—the range is marked for both as if these were one track. Previewing the range is important because it allows a user to see if range of selected tracks fits onto the attached MIDI keyboard. i.e.
  • This option allows merging together a number of selected tracks into one. This operation does not alter the content of the original file, instead it is performed in the computer's memory on the file loaded.
  • Deletes selected track(s) from memory Deleted track(s) are not deleted from a song file.
  • this option is enabled. Executing this option splits a single track into two tracks with musical content arranged separately for both hands. I.e. The first track holds musical notes to be played with a left hand, and the second track is to be played with a right hand.
  • This option transposes selected track(s) by semitones either up or down.
  • This section allows a user to configure MIDI output ports and channels for any track of the file loaded. For instance, a user might prefer to hear an accompaniment track #2 on the 11 th channel of a synthesizer located on the 3 rd MIDI port.
  • This switch enables or disables the audible metronome heard during the game play.
  • This section allows for selecting a challenge from all the challenges that are built into the game, and are part of the Career Mode, as discussed earlier. By allowing this, users can flexibly practice different challenges on any song they wish to.
  • This option allows for a selection of a tempo that is used during the game play. Part of this option is a switch called “MIDI tempo: On
  • This option allows for adjusting the height of the hit window of the game.
  • Opacity values range from 0% to 100%.
  • This switch enables or disables the visual measurement that scrolls along with the game objects.
  • This switch turns on or off on-screen game statistics, score, etc.
  • This option allows for selecting the size of the on-screen keyboard used in the game play. Users can select between 2-octave, 4-octave or 5-octave versions of keyboards. Automatic keyboard size selection is also available. The game will support three keyboard sizes: 2-, 4-, and 5-octave.
  • This option allows for selecting the starting key on the on-screen keyboard. This is very helpful in calibrating the physical MIDI keyboard with the on-screen keyboard if the two are of a different size.
  • the selections are: C0, C1, C2 . . . C11.
  • Enabling this option will gradually increase or decrease the tempo of the game play to the tempo specified in a loaded song file. For instance, by starting the game play at a low tempo, users can practice a piece of music at a slowly, but constantly increasing tempo.
  • Enabling this option will gradually decrease the height of the hit window during the game play. For instance, by starting the game play at a hit window wide open, users can practice a piece of music with a slowly, but constantly increasing precision.
  • lyrics will appear on the screen.
  • This option enables or disables auto-saving of all alternations to a currently loaded song. If auto save song is enabled, a user is prompted for a new file name for a song. Songs are saved into an .amm internal song file format.
  • a user chooses the clefs (tracks) to be merged.
  • This option is merely separating out the text into measures, for someone to sing along without the notation being on the same sheet.
  • “Fake” books are collections of popular songs with the lyrics and the chord names (Gm7) for jazz players, and maybe the lead melody, and they would improvise the rhythm and harmonic parts, using the melody and chord structure as a skeleton.
  • MidiNotate software extracts the lyrics, lead melody, and chord names from a midi file, and allows for printing it in this manner.
  • Chord names have multiple ways of being described. An example might be A Major. It could be written simply A or it might be written as A maj.
  • Game play is the actual game.
  • keyboard shortcuts linked to different options which can be invoked during the game play without stopping the flow of the game.
  • Other options are possible. These options are:
  • tempo adjustment is not available during the game play in tempo challenge.
  • a mark is placed onto the game field. Users are able to scroll up and down the entire song from within the freeze menu and see where they have missed.
  • the invention is capable of transforming MIDI files into a musical notation to be displayed on the screen during the game play.
  • Examples of software suitable for implementing this function can be found at:
  • the invention features a number of game objects, also known as Sprites. All game objects will feature 32-bit color and alpha-blending support.
  • the game objects are two dimensional, but drawn on the screen using 3D techniques in order to take advantage of hardware accelerated 3D drawing. This will allow for an easy scaling, rotating and alpha-blending.
  • the on-screen piano keyboard comes in three different sizes: 2-, 4-, and 5-octave. Depending on the keyboard size selected, size of the game object will change accordingly to keep the width proportional to the width of the on-screen keyboard keys.
  • the game rewards users with points for correctly hit musical notes, playing with different dynamic options enabled, and for playing different challenges, and so on.
  • Freeze Menu can be invoked from within the game play only. It will allow for alternation of some of the game parameters, making annotations on the screen and saving a currently played song into an internal file format along with all annotations, highlighting and fingering information.
  • the freeze menu is invoked by pressing an ESC key on the keyboard from within the game play. Once ESC is pressed the following happens: the entire game freezes, vertical scrollbar appears on the screen to allow scrolling through the entire song both, back to the parts already played, and ahead. Note that scrolling, annotating, highlighting or fingering is not available when the game screen is in the middle of a rotation. Once the game is frozen, a translucent help box is displayed with directions as to what can be done on the screen. For instance, a typical text would read: “To create an annotation on the game screen, click on an empty space”.
  • Annotations are placed on the screen in a form of speech balloons. There are a number of balloon shapes to choose from.
  • a user would click on an empty space on the screen. Once clicked, a popup menu will appear allowing for selecting the shape of the speech balloon and the direction of its arrow-pointer. The text would be entered then through a text edit box. Annotations can be removed from the game screen.
  • freeze menu options are dependent on the game mode. For a preferred availability of freeze menu options in different game modes please see Attachment B. Freeze menu options are listed below. It is worth noting that a significant number of these options are a subset of Starting Options, which have been defined in section C above. Definitions of already defined options are omitted here.
  • Prompts for a file name saves a current song into an .amm file.
  • Game setup can be invoked from the main menu.
  • This menu option allows selecting between two different sets of menu options in the game.
  • Advanced set is designed for users who want to take the game to the fullest and require maximum flexibility, whereas beginner set eliminates a number of menu options that are simply too complex for an average user, or a child.
  • This option allows adjusting the gamma value of the screen.
  • This menu option allows for selection of the MIDI input port for the game.
  • the game will enumerate all possible MIDI ports on the system and make them available for selection.
  • This menu option allows for selection of the MIDI output port for the game.
  • the game will enumerate all possible MIDI ports on the system and make them available for selection.
  • Game detail level enables and disables some of the CPU intensive game visual effects. Small level of detail will make it possible to run the game on a low-end computer at the price of visual effects and overall eye-candy.
  • SFX Volume option allows for an adjustment of the overall volume level of all .wav sounds played in the game (PCM channels of the sound card), whereas Music Volume allows for an adjustment of the overall volume level of the internal sound card synthesizer. Both sliders can be assigned values from 0% to 100%.
  • This option will allow one to test a user's MIDI keyboard to determine whether it is velocity sensitive or not. A user is asked to strike a key on the MIDI keyboard twice, first gently, and hard the second time. Based on the difference in value of the received MIDI velocity message, the program will determine if a user has a touch-sensitive keyboard.
  • the CD-ROM contains a plurality of directories and files as shown in the following printout of directories from the CD-ROM.
  • the file extension description of most files adequately specifies to one skilled in the are how to open them.
  • the files can be opened using Microsoft Word, Microsoft Excel, or an adobe Acrobat Reader.
  • the .vsd file can be read using Visio.

Abstract

A musical keyboard is connected to a computer. The computer implements a graphical user interface for teaching users to play the musical instrument. A computer readable music file, such as a MIDI file, is used to drive the creation of game objects that travel from a point of origination along a path toward a key of a virtual keyboard. In one form, when a user presses a key of the musical keyboard within a certain time window of arrival of the game object at the corresponding key of the virtual keyboard, the user is awarded with an audio presentation, a visual presentation and/or with game points. In a more structured learning mode, the game can be played with selectable, progressively more difficult challenges that the user masters on the road to proficiency.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • This application claims priority from and is related to U.S. Provisional Application Ser. No. 60/347,554, filed on Oct. 20, 2001, entitled “AN INTERACTIVE GAME PROVIDING INSTRUCTION IN READING MUSICAL NOTATION”, by inventor Hal C. Salter. This application also claims priority from and is related to U.S. Non-Provisional application Ser. No. 10/273,353, filed Oct. 18, 2002, entitled “INTERACTIVE GAME PROVIDING INSTRUCTION IN MUSICAL NOTATION AND IN LEARNING AN INSTRUMENT”, by inventor Hal C. Salter. This application further claims priority from and is related to U.S. Non-Provisional application Ser. No. 11/411,835, filed on Apr. 27, 2006, entitled “INTERACTIVE GAME PROVIDING INSTRUCTION IN MUSICAL NOTATION AND IN LEARNING AN INSTRUMENT”, by inventor Hal C. Salter. The contents of the provisional application, non-provisional application, and continuation application are hereby incorporated herein by reference in their entirety.
  • COPYRIGHT NOTICE
  • At least a portion of the disclosure of this patent document contains material that is subject to Copyright protection. The Copyright owner has no objection to the facsimile reproduction by any of the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all Copyright whatsoever.
  • CROSS REFERENCE TO CD-ROM APPENDIX
  • This application contains an Appendix comprising (1) CD-ROM containing specifications of computer programs used to carry out the invention disclosed herein. The contents of the CD-ROM are described in more detail in paper Appendix A attached to this document. The contents of the CD-ROM are incorporated into this disclosure by reference in their entirety as if they were set forth completely within the text of this application.
  • FIELD OF THE INVENTION
  • This present invention relates generally to an interactive game, and more particularly to a process for interpreting or compiling MIDI or other standard music files into an interactive game that instructs its used in musical notation while playing the game.
  • BACKGROUND
  • MIDI technology has been a stable source of piano and music sequencing programs for editing, printing, playing and composing music for almost twenty years. Some early, relatively primitive music and piano tutorial programs, as well as some games, have resulted from this technology. For example, some types of computer aided instruction, including piano instruction, have been used in the home market for several years.
  • In the area of music instruction, the computer has been used, for the most part, to provide a student with an interactive view of musical notation, where the student can press a key on the accompanying electronic piano and it will light up the note, or the note will light up and a drawing will indicate which key he should press. Other systems have a series of lights or fingering illustrations to attempt to communicate to the user which key should be played next. These programs and systems, while allowing the student to practice and obtain feedback, generally fail to involve musical novices sufficiently, especially with regard to rhythm. Several typing tutorials have enjoyed great success using such methods, but piano and music tutorials have not been as successful, due to several significant problems inherent with the subject.
  • A major problem with these prior art learning systems is that they fail to take into account several important differences between learning to type and learning to play music. The first and most obvious difference is that musical notes cannot merely be played in a correct sequence, but must be played in a rhythmic pattern in relation to the notes before and after it. The correct timing of the notes is one of the most difficult aspects to teach in music, by any technique; and up to now few applications have been able to adequately address this challenge, and then only in isolation from other concepts. The second challenge is that most programs start with musical notation to teach a song to the user. Reading and understanding musical notation is a bit like reading Beowolf in the Old English, and it's roots are just as old. The five line staff originally represented a monk's hand in the Middle Ages, and has been adapted to deal with all the musical innovations since.
  • There is also the problem of a vertically oriented notational system originally designed to convey pitch for voice (hence the concepts of higher and lower). But on the piano, pitch is experienced horizontally (i.e. perpendicularly to the notation—left to right instead of lower to higher). This difference, while conceptually easy, is a difficult transition for students to make, both visually and experientially. The combined difficulty is such that fewer than 20 percent of all piano players are estimated to be able to read music fluently.
  • For the novice player, then, there are at least four additional challenges to learning the piano compared to learning to type. They must learn the keys to the piano, they must learn musical notation, the notation is perpendicular to their piano, and they must hit the right key(s) at the right time. The frustration at these simultaneous challenges causes most users to quit, and makes the task seems more like work than play.
  • Examples of prior patents in the general area of music instruction are outlined below. Each of these references is incorporated by reference in their entirety.
  • U.S. Pat. No. 4,416,182 to Wise, et al. discloses a keyboard teaching device for the self-instruction of a student of keyboard musical instruments. They system enables the student to correlate the positions of the keys on a musical instrument keyboard with the positions of the notes on a musical scale. A keyboard having a plurality of keys corresponding to the notes of a musical scale generate a first set of control signals in an initialization or set-up mode and key-note correlation signals in an instruction or game mode. There is at least one storage element storing a predetermined combination of logical signals providing a source for a pseudo random sequence of one or more notes over a predetermined range in the game mode. The system generates a second set of control signals in response to the actuation of one or more keys in either game mode and a mechanism for generating audio tones and displaying video images in accordance with the first and second sets of control signals enables a student to visually and audibly check his or her selection of one or more keys.
  • U.S. Pat. No. 5,183,398 to Monte, et al. discloses an apparatus and method for instruction of a student which includes interactive guidance of the student through a series of lesson frames. This disclosure provides the student with a keyboard having a plurality of keys corresponding to the notes of a musical scale and generates a key relation signal in response to each depressed or released key. A video display and an audio tone generator associated therewith enables the student to respond to the visually displayed images and audio tones by selecting one or more of the keys on the keyboard. The student is presented with a lesson frame representing an instructional activity requiring a response by the student on the keyboard. The student keyboard response is compared with a performance standard and an absolute performance evaluation result in generated. The absolute performance evaluation result is compared with an acceptable achievement level for the particular instructional activity and a next frame selection signal is generated. A next frame is selected for presentation to the student based upon the next frame selection signal.
  • U.S. Pat. No. 4,997,374 to Simone discloses a teaching device that includes a changeable two channel prerecorded program source, and a console unit including a work booklet. The first channel of the program source includes an audio program comprising a series of spoken words which are audibly reproduced by the console unit and the second channel includes a series of control signals which are operative for actuating lights adjacent prespecified words in the work booklet. The operation of the console unit is coordinated with the audio program so that lights are actuated adjacent to the words in the work booklet as the same words are audibly reproduced by a console unit. One embodiment of the device further includes a plurality of depressible user response buttons on the console unit for indicating responses to questions presented in the audio program.
  • U.S. Pat. No. 4,781,099 to Koike discloses a musical quiz apparatus that presents a question chord in sound and a trainee answers by depressing the keys of the chord constituting notes on the keyboard. The apparatus is capable of generating a plurality of different chord data respectively representing chords, and generates a question chord datum one at a time more or less randomly selected from among those different chord data and produces sounds of notes which constitutes a chord designated by the question chord data. When the answer is correct, points are added up and a next question chord is presented.
  • U.S. Pat. No. 5,392,682 to McCartney-Hoy discloses a computerized musical keyboard and a method for using same to play or to learn to play a piano. The computerized musical keyboard includes a piano keyboard connected to a computer. The computer is programmed to select from a music module a piece of music to be played on the piano and to generate a signal indicating the proper keys to be played, the correct sequence in which the keys are to be played, and the hand and finger to be used in striking each key, in order to play on the piano the piece of music selected.
  • U.S. Pat. No. 5,107,743 to Decker discloses a piano teaching aid having a panel designed to fit over the keys on an existing keyboard so that lights mounted on a panel having more than one color or shape may be located directly above the piano keys to be played. The lights have more than one color or shape in order to distinguish the hand which the user will use to play the piano. The panel also includes a finder window which displays an alphanumeric code which corresponds to a like code appearing next to the score of music to be played. A foot pedal advancing mechanism is used whereby the user can control the speed which the lights display the keys to be struck, using the Foot Pedal to advance one action. The display also can show the music to be played at a tempo set by the user automatically changing from action to action without using the pedal. The panel articulates so that it may be stretched in one or more places so that it can fit over various dimensions of keyboards without interfering with keys to be played.
  • U.S. Pat. No. 4,331,062 to Rogers discloses an apparatus for visually displaying music notes on a note display panel mounted on an electronic piano with a support arm. The electronic piano has a keyboard electrically coupled to an electronic circuit operable to produce an audio output in accordance with the depression of one or more keys. The arm is rotatable in a mount attached to the piano for movement about a first upright axis. A first motion limiting unit attached to the mount and arm limits the rotation of the arm about the piano. A second motion limiting unit attached to the panel and this arm limits rotation of the panel about an upright axis relative to the arm. A modified structure has the arm fixed to the piano. The panel has grand staff indicia coordinated with vertically disposed first light mechanisms corresponding to chord note information with a second light mechanism diagonally corresponding to ascending note scale information. A keyboard representation is located below the staff indicia. A third light mechanism associated with the piano key indicia of the keyboard representation is coordinated with the second light mechanism to provide visual information as to the keys depressed on the electronic piano. An electric circuit having on-off switches electrically couples the electronic piano with the first, second, and third lights so that the lights can be selectively operated. The electronic circuit has a switch assembly having a plurality of key signature switches operable to coordinate the first, second, and third lights with the piano keyboard in accordance with the key signature of the music that is played.
  • U.S. Pat. No. 4,366,741 to Titus discloses an electronic piano having a keyboard and an electronic piano circuit connected to a micro-processor used to control a CRT device to provide a video note display concurrently with the depression of one or more keys. A keyboard representation located adjacent the screen of the CRT device is associated with lights used to indicate the key or keys that are played. Manually operated controls cooperate with the micro-processor to allow the back clearing of the screen one note at a time, remove all the notes, retain all the notes, indicate sharp or flat mode of each note, and indicate the duration that a key is depressed by elongating the note on the screen. A metronome unit is used with the micro-processor to provide a visual beat marker on the screen that sequentially moves across the screen. A movable frame connects the CRT device to the piano.
  • U.S. Pat. No. 5,864,868 to Contois discloses a computer system and method for controlling a media playing device. The system provides a user interface for allowing a user access to media pieces stored in a media database. The interface is also for controlling a media playing device, like a player piano or movie playing video device, that is coupled to the computer to play the accessed or selected piece of media. In one embodiment there is a computer interface that allows a user to display only music that relates to a selected category, like jazz or classical music. Another embodiment allows the user to direct the media playing device to automatically play selected music pieces that are related to a selected music category. Another embodiment allows a user to direct the media playing device to automatically play selected music pieces that are related to the selected music composer or artist.
  • U.S. Pat. No. 6,204,441 to Asahi et al. discloses techniques for displaying musical information and particularly for visually displaying musical notes, beats and tempos using personal computers or game devices that run musical software programs. The disclosure teaches the use of different colors and different brightnesses to distinguish certain types of musical notation. It has a display screen which shows both base and treble clefs as well as the keyboard and timing indication.
  • U.S. Pat. No. 6,388,181 to Moe shows computer graphics animation, used with a live video interactive method for playing keyboard music while the user guides his fingers to the keys targeted by the animation with each key to be struck within one beat of time is designated by a colored “sprite”.
  • U.S. Pat. No. 6,066,791 to Renard et al. shows a system for instructing the playing of a musical instrument, displaying an image on the display device, and instructing the student to focus on the image while preferably using a musical instrument to play the notes on the staff.
  • U.S. Pat. No. 5,540,132 to Hale shows techniques for teaching musical notation to children. Each note is associated with the distinctly identifiable color which is then associated with an object which naturally occurs in this color. They utilize cartoon characters which apparently enhances association within the mind of the child.
  • U.S. Pat. No. 6,337,433 to Nishimoto et al. shows an electronic musical instrument having performance guidance function, performance guidance method, and storage medium storing a program therefore with a plurality of display devices arranged in association with the performance operating elements, respectively, each comprising a pair of display elements corresponding to left and right hands of the player, respectively.
  • U.S. Pat. No. 6,284,961 to Kimmel, Jr. shows a system of musical notes with the notes being associated with a color and utilizes stickers for application to the keys of the musical instrument to correspond to the colors of the note which it plays.
  • SUMMARY OF THE INVENTION
  • The invention is related to a hand-eye coordination game which has the purpose of teaching one how to read musical notation and how to play a musical instrument such as the piano.
  • The invention simplifies musical notation into a grid, with pitch moving left (lower) to right (higher), parallel with the piano keyboard layout, and the dimension of time moving vertically.
  • In one embodiment, in the beginning phases of the game, the player will see game objects, representing notes of a song, rising from near the bottom of the screen toward the virtual piano keyboard, and their vertical relationship to each other shall be a representation of relative musical time. As these game objects approach the virtual piano keyboard, the corresponding piano key along that path is visually obvious, and the objective of the player is to hit the corresponding key on a musical keyboard at precisely the time the game object is located within a predefined hit window vis a vis the virtual keyboard. If the player strikes the correct key at the correct time, audio and visual feedback will reward the player.
  • The player's main task in the invention is to hit the corresponding piano key for as of the many moving objects as possible at the right time, repeating the challenge until they have reached a certain percentage of correctness and move up to the next level of complexity, and in doing so, subconsciously learn to play a song, and in the later levels, to recognize and read the sheet music.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The objects, features and advantages of the system of the present invention will be apparent from the following description in which:
  • FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the invention.
  • FIG. 2 is a diagram showing a virtual keyboard, game object trajectories and an exemplary relationship of those to a treble clef of a standard musical notation.
  • FIG. 3 shows a block diagram of exemplary hardware and software modules for carrying out the invention.
  • FIG. 4 illustrates the format of a MIDI file.
  • FIG. 5 illustrates the format of a header chunk of a MIDI file.
  • FIG. 6 illustrates the format of a track chunk of a MIDI file.
  • FIG. 7 illustrates the format of META events from a MIDI file.
  • FIG. 8 is an example of a portion of a MTrk chunk of MIDI data together with exemplary semantics.
  • FIG. 9 illustrates an exemplary flow of data illustrating MIDI messages for the sequential playing of three notes.
  • FIG. 10 is a flow chart of an exemplary process for interpreting the MIDI data stream of FIG. 5 for generating game objects for display.
  • FIG. 11 is a block diagram showing an exemplary arrangement of game modules in accordance with one aspect of the invention.
  • FIG. 12 is a block diagram of an exemplary computer that can be used to implement various aspects of the invention.
  • DETAILED DESCRIPTION OF THE INVENTION
  • FIG. 1 is a block diagram of a hardware architecture for carrying out some aspects of the invention. FIG. 1 illustrates a computer 100 comprising a processing element (CPU) and a display. A typical implementation would be a personal computer commercially available from a number of sources. Details of the internals of computer 100 and its use in a network environment in FIG. 12.
  • A piano keyboard, 150, preferably having the capability of generating controlling data in accordance with the MIDI standard is connected to the computer through the user standard MIDI interface.
  • FIG. 2 is a diagram showing an exemplary virtual piano keyboard, game object trajectories and an exemplary relationship of those to a treble clef of standard musical notation. A virtual keyboard is an object that is shown on the screen of the computer's display as part of a graphical user interface. The virtual keyboard will comprise a plurality of individual keys 200 i corresponding to the white keys of a piano and a plurality of keys 200 j corresponding to the black keys of the piano. Thus, in this embodiment, the virtual keyboard represents the standard arrangement of keys on a keyboard such as a piano. Also associated with each key of the virtual keyboard is a trajectory 210 i corresponding, in this example, to only the white keys of the piano keyboard. One of the white keys and an associated trajectory correspond to each of the notes of an octave of musical notation, in this case, starting with middle C on the left and progressing up the musical scale until reaching the C above middle C. Trajectories shown in bold correspond to those note found on a line of the treble clef in this example.
  • From this view, it is clear that pitch increases as one progresses up the musical scale from left to right as shown in this view. However, in standard musical notation, middle C begins on the line below the first line of the standard treble clef shown in FIG. 2 and the C above middle C is shown in the top most space of the standard treble clef. Thus, musical notation progresses in pitch on treble clef from bottom to top starting with low pitch value and increasing to higher pitch value. However, when viewed from a keyboard, pitch increases from left to right rather than from bottom to top. As noted above, this difference of representation creates at least one factor of difficulty in learning to play the piano.
  • As will be described more hereinafter, game objects are introduced at a location opposite the virtual keyboard along the trajectories 210 i. Game objects then progress in animated fashion along the trajectory from near the bottom to the point where the game object encounters the key of the virtual keyboard 200 i. There is associated with each of the trajectories a traversal time so that if three notes were introduced sequentially one after the other, they would become visible at the bottom of the trajectories appearing in a sequence, and progressing for the duration of the traversal time from bottom to top where they intersect their respective keys at respective instants of time displaced from one another.
  • One object of the interactive game is for a student to press the key of a (preferably MIDI) keyboard substantially at the same instants that the game objects intersects they key of the virtual keyboard on the graphical user interface. As the user plays the game, the user develops a certain facility for mapping the visual expression of the notes from a musical file which drive the generation of the game objects into actual fingering positions on a real keyboard which interfaces with a virtual keyboard. The game detects when the user presses the correct key at the proper moment.
  • FIG. 3 shows a block diagram of exemplary hardware and software modules for carrying out various aspects of the invention. Although this particular embodiment of the invention is described with respect to the use of MIDI files and music formatted in accordance with a MIDI standard, other formats for music are well known and can be utilized in alternative embodiments of the invention.
  • In this exemplary embodiment, one or more files 300 formatted in a MIDI format are stored in, for example, a library, or downloaded in real time and fed to interpreter 310 where the MDI data file is interpreted and translated in to commands which drive the game object generation module 320 which causes the display of game objects on the graphical user interface on the display screen and provides audio output 370 to the speakers of the computer. Conveniently, a graphics engine 350 and an audio engine 360 simplify the translation of game objects into audio and visual components driving the visual display and the audio output. The game object generator has access to a library of objects 330 and to a set of game control parameters 340. Each of these modules are described in more detail hereinafter.
  • FIG. 4 illustrates the format of a MIDI file. As noted above, the MIDI file format is utilized for purposes of illustrating the invention, but the invention is not limited hereto. An example of another file suitable for carrying out the invention would be that of MPEG-4.
  • An understanding of a MIDI file and the operation of the MIDI protocol will greatly simplify understanding of the invention.
  • The MIDI File Format
  • MIDI files are structured into chunks. Each chunk consists of a 4-byte chunk type, a 4-byte indication of length indicating of the length of the bytes contained in the data field.
  • There are two types of chunks, namely header chunks (which have a chunk type of “MThd” (410)) and track chunks which have a chunk type of “MTrk” (420).
  • A MIDI file consists of a single header chunk followed by one or more track chunks.
  • FIG. 5 illustrates the format of a header chunk of a MIDI file. The header chunk comprises three fields, namely, chunk type, length and data. For a header chunk, the chunk type is MThd. The length field contains the length in bytes of the chunk data part. The “format” portion of the data field includes the MIDI file format which can be only formats 0, 1 and 2. The “tracks” portion of the data field as a binary number indicating the number of track chunks contained in the MIDI file. The “division” portion of the data field of the header chunk defines the default unit of “delta-time” for the MIDI file. If the most significant bit of the “division” field is a 0, the remaining 15 bits indicate the number of “ticks per quarter note” to be utilized in representing and reproducing the music. If the most significant bit is a logical 1, then there are two components indicated by the remaining 15 bits. Bits 8-14 would indicate the number of frames per second (indicated as a negative number) and the least significant 8 bits represents the number of ticks per SMTPE frame.
  • MIDI files come in three variations. Format 0 contains a single track. Format 1 contains one or more tracks which are all played simultaneously. Format 2 contains one or more independent tracks which can be or are played independently of the others.
  • FIG. 6 represents the format of a track chunk of a MIDI file. It comprises a chunk type which, by definition is a MTrk type and a length field which indicates the length of the data portion in the track chunk. The data portion of a track chunk comprises two elements. The first is a “delta_time” portion and “event” portion. delta_time is the number of “ticks” from the previous event and is represented as a variable length quantity. There are three types of events that are defined within the standard. They are a “MIDI event”, a “SYSEX event” and a “META event”. There are no explicit delimiters between the “delta_time” and “event” instances. This is possible because both fields have clearly defined lengths.
  • A MIDI event has any MIDI channel message. These include channel voice messages and channel mode messages. Messages other than MIDI channel messages to be included in a MIDI file can utilize the SYSEX event.
  • Most system exclusive messages are quite simple and are sent as a single packet of bytes, starting with F0 and ending with F7. However, some system exclusive messages are used to control device parameters in real time. Two different types of SYSEX events are defined to accommodate the different usages.
  • META events are used for things like track-names, lyrics and que points, which don't result in MIDI messages being sent, but are still useful components of a MIDI file.
  • MIDI events have the general form shown in FIG. 7. MIDI events are preceded with the hexidecimal notation FF followed by a type field, a length field and a data field. The type field is a single byte specifying a type of META event. The length field contains a number of bytes of data following that field. The data field includes 0 or more bytes of data.
  • A number of META events have been defined in the standard that make implementation of the invention easier. These include the following:
  • A “sequence number” is an optional event which must occur only at the start of a track before any non-0 delta_time.
  • This is typically utilized to identify each track. A text event is utilized for annotating a track with arbitrary text. A Copyright notice event can be utilized where a Copyright notice is represented in ACSI text. It should be the first event on the first track of a MIDI file.
  • A sequence/track name provides the name of a sequence or a track in the file. An instrument name, provides the description of the instruments used on the track. A lyric event provides the lyrics for a song. Normally, each syllable will have its own lyric-event, which occurs at the time the lyric is to be sung.
  • A marker event marks the significant point in the sequence such as the beginning of a verse. A que point is utilized to include queues for events happening on-stage, such as “curtain rises”, “exit”, and the like.
  • An end of track event must be utilized to give the track a clearly defined length. This is essential if the track is looped or concatenated with another track.
  • A set tempo event sets the tempo in microseconds per quarter note. This means a change in the unit-length of the delta_time tick. The default tempo is a 120 beats per minute. A SMTPE offset event specifies the SMTPE time at which the track is to start. A time signature representing the standard time signature for a piece of music, such as 3/4 or 6/8 or 2/2. A key signature event can specify the number of sharps or flats and a major or minor flag. Finally, a sequence-specific META event allows a manufacture to incorporate sequencer specific directives into a MIDI file.
  • FIG. 7 illustrates the format of META events from a MIDI file.
  • FIG. 8 is an example of a portion of an MTrk chunk of MIDI data together with exemplary semantics. As shown in FIG. 8, an MTrk chunk is introduced with an MTrk identifier 800 followed by a length field 810. The plurality of ordered pairs of delta_time and event fields 820-1 through 820-j then follow which represent the individual MIDI messages associated with the MTrk chunk.
  • The MIDI protocol consists of messages which are designed to allow synthesizers and sequencers to communicate what-sound-to-play information. A typical MIDI message comprises three components. The first component begins with a hexidecimal 9 is followed by an identification of one of sixteen MIDI channels having a value of O-F. The second component is a two byte sequence representing the key on the device that has been pressed. This corresponds to the notes of a keyboard that has been pressed in the case of a piano-type keyboard. The two byte value ranges from 00-7F. The third component of a MIDI message is the velocity component which specifies the velocity with which the key was pressed or released. It, too, ranges in value between 00 and 7F. Thus, if a musician pressed middle-C key on a keyboard, the keyboard would send a “note-on” message comprising: 90 3C 40. When the musician releases the key, the corresponding “note-off” message would comprise: 80 3C 33. In this case, the key was released more slowly than it was pressed as indicated by the velocity indication 33 in the release message (compared with the attack velocity 40).
  • Other MIDI messages may include a program (instrument) change, a pitch blend message, a control change message (e.g. pedal/switch foot change of state) and timing clock message.
  • MIDI messages are all one way. There are no acknowledgement messages sent from the receiver back to the transmitter. If a MIDI device does not know what to do with a message, it will ignore it. MIDI messages which are specific to a MIDI channel are referred to as channel messages. MIDI messages which affect the entire MIDI system or an entire MIDI device are known as system messages. Channel and system messages are further divided into several classes. The channel voice messages are messages which start, alter or stop a sound or sounds being played. Channel mode messages effect the entire channel. System realtime messages are those used by sequencers to regulate and synchronize timing. They do not contain data bytes. System common messages include messages such as song position pointer, song select and the like. System exclusive messages are generally used for device specific extensions to a MIDI protocol.
  • Since MIDI messages are sent and interpreted in realtime, it is desirable to reduce the volume of data that must be sent. For ordinary note-on and note-off messages, it is quite common for several notes to be turned off and on more or less at the same time. In such cases, it is possible to send a single status command, such as note-on followed by a plurality of “note identifier, velocity” pairs without repeating the same status byte for each note that is turned on or off. This reduction in the transmission of status bytes is known as “running status” within MIDI messages. Only the data bytes that change are sent.
  • Against the background of the MIDI File and protocol, an exemplary operation of that invention will now be described.
  • FIG. 9 illustrates an exemplary flow of data illustrating MIDI messages for the sequential playing of three notes. In FIG. 9, two MIDI messages are shown per line. Each is introduced with a delta_time field. Thus, the six messages shown in FIG. 9 are preceded with delta_time increments of 00, 08, 10, 18, 20 and 28.
  • The delta_time increments are followed by a MIDI command sequence. The sequence 92 indicates the situation where a note is to be turned on. Following each 92 message is a message with a command field of 82 which indicates that the note is to be turned off. Following each command is a note identifier. Which in the case of the first line indicates the note C is to be turned on and off. In the case of the second and third lines, the notes identified are the note E and the note G, respectively. The last component of each message is a velocity component. In the first line, the velocity 44 is a measure of how fast the key is pressed downwardly (e.g. attack velocity). In the following message, where the key is being released, the velocity 40 indicates that the key is released with a velocity 40.
  • FIG. 10 is a flow chart of an exemplary process for interpreting the MIDI data stream of FIG. 5 as part of generating game objects. When a MIDI message is received representing a note on object of the form <delta_time><Command><Note><Velocity> (1000) optionally followed by a MIDI message representing a note-off property of similar format (1010). At time <delta_time> less traversal time, an object is launched on <Note> trajectory with optional length property of <delta_time (off)>-<delta_time (on)> ticks (1020). In this form, one can see that steps 1000-1020 represent an interpretation of the incoming MIDI messages followed by the generation of a game object. Messages are received and interpreted on an ongoing basis in accordance with the steps 1000-1020. A window is defined within τ units of time of impact with the virtual keyboard (1030).
  • Once the object is launched on the note trajectory, the object is moved along the trajectory towards the virtual keyboard with each N ticks of the MIDI clock (1040). If MIDI keyboard input is received during an open window and if the value of the MIDI keyboard input equals the MIDI note value of the object about to impact the virtual keyboard, video and audio reward presentation (routines) are activated (1060). Otherwise, a video and audio sequence representing a failure can be activated. This process is repeated for each tick of the MIDI clock in an iterative fashion as indicated by the arrow going back to the top of step 1040.
  • Returning to FIG. 3, item. 330 represents a library of objects which allow the game to be modified by the substitution of various types of objects for corresponding different flavors of the game. Examples of the types of objects utilized are given hereinafter. Also shown in FIG. 3 is a set of parameters 340 which allow a user to customize the performance of the game or the behavior of the game in certain instances.
  • The audio engine 360 is preferably based on Microsoft® DirectSound 8 application programming interface. The audio engine generally has the ability to play short .WAV files through the PCM channels of a sound card. The sound engine is implemented as a class with constructors, destructors, loading of a .WAV file and playing of a .WAV file. The sound engine is used primarily for playing sound effects associated with menu buttons, clicks, switches and game play feedback sounds. However, the invention could also work with other software engines.
  • The graphics engine 350 works in conjunction with the library of objects 330 and the game object generation module 320 to produce graphical objects on the screen that constitute respective implementations of the game objects utilized to play the game. The invention is capable of loading and playing three types of song files: MIDI, MIDI Karaoke, and .amm. The later is an internal song format described more hereinafter. Standard MIDI files were described above. The MIDI Karaoke file is essentially a standard MIDI file with song lyrics built into one of the tracks according to a certain format. The invention supports the .KAR file format which is a prevailing file format in the shareware/public domain markets.
  • An .amm file is very similar in structure to a standard type 1 MIDI file. However, the .amm format file will also hold annotations, highlighting and fingerings. Annotations are simple text notes that are stored on a song's timeline. These annotations can hold valuable information to a user and can be displayed during the game play along with other scrolling game objects. They may be stored as text in the internal file, but on the screen these are shown in the form as speech balloons with or without arrows pointing to other game objects.
  • Some notes in an .amm file can be highlighted. This is essentially a flag that tells the system that a note should be displayed with an enhanced video effect such as a halo effect around it. Fingerings are little clues to a user as to which finger should be used to play a note on a piano keyboard. Fingerings are assigned to individual notes and hold a number in a range from 1 to 5. During game play, game objects that have fingerings assigned will scroll along with the number placed on them.
  • FIG. 11 is a block diagram showing an exemplary arrangement of game modules in accordance with one aspect of the invention.
  • When a game is first loaded, a splash screen 1100 is displayed. This is centered on the desktop and enables the user to click on icons or links that would take them to the sponsoring company's website where they can download new songs for the game. Also, there may be a link to HTML documentation for the game.
  • The Welcome Screen 1110 allows four options. First, it allows selection of a main menu which is described hereinafter. Second, it allows access to an instant play mode of operation, described in conjunction with item 1140 hereinafter, which assumes all previously set up configurations of the last song played in the game. A third option from the main menu takes one to the credits screen where game credits are revealed in a visual way. Finally, the main menu 1120 has an option to exit the game and return to the operating system.
  • From the main menu 1120, there are three options. If a quick play option is selected, the user goes directly to the quick play mode described in conjunction with 1140 where the user can freely load any song, select a number of options and commence the practice. No profile needs to be selected or created. In the second main menu option, the career mode, a user will create a profile and will face a number of challenges, predefined within a number of levels and in a number of different piano methods. These are discussed in detail hereinafter in conjunction with career mode 1150.
  • Also from the main menu, a game set up 1130 may be selected. In game set up, a number of game objects can be configured as described more hereinafter.
  • Instant play 1140 allows the user to play the game without the hassle of going through start up options. While previous starting option parameters are assumed, including the last song played if the game was initially installed and no songs have been played yet, a default song is assumed. Instant play has no menu, it is just a short cut to the game play.
  • FIG. 12 is a block diagram showing details of an exemplary computer that can be used to implement various aspects of the invention. The description of the invention which follows is exemplary. However, it should be clearly understood that the present invention may be practiced without the specific details described herein. Well known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention. At least portions of the invention are intended to be implemented on or over a network such as the Internet. An example of such a network is also described in FIG. 12.
  • FIG. 12 is a block diagram that illustrates a computer system 100 upon which an embodiment of the invention may be implemented. Computer system 100 includes a bus 102 or other communication mechanism for communicating information, and a processor 104 coupled with bus 102 for processing information. Computer system 100 also includes a main memory 106, such as a random access memory (RAM) or other dynamic storage device, coupled to bus 102 for storing information and instructions to be executed by processor 104. Main memory 106 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 104. Computer system 100 further includes a read only memory (ROM) 108 or other static storage device coupled to bus 102 for storing static information and instructions for processor 104. A storage device 110, such as a magnetic disk or optical disk, is provided and coupled to bus 102 for storing information and instructions.
  • Computer system 100 may be coupled via bus 102 to a display 112, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 114, including, for example, alphanumeric and other keys, is coupled to bus 102 for communicating information and command selections to processor 104. Another type of user input device is cursor control 116, such as a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 104 and for controlling cursor movement on display 112. This input device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows the device to specify positions in a plane.
  • Computer system 100 operates in response to processor 104 executing one or more sequences of one or more instructions contained in main memory 106. Such instructions may be read into main memory 106 from another computer-readable medium, such as storage device 110. Execution of the sequences of instructions contained in main memory 106 causes processor 104 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.
  • The term “computer-readable medium” as used herein refers to any medium that participates in providing instructions to processor 104 for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 110. Volatile media includes dynamic memory, such as main memory 106. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise bus 102. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infra-red data communications.
  • Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punchcards, papertape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.
  • Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to processor 104 for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to computer system 100 can receive the data on the telephone line and use an infra-red transmitter to convert the data to an infra-red signal. An infra-red detector can receive the data carried in the infra-red signal and appropriate circuitry can place the data on bus 102. Bus 102 carries the data to main memory 106, from which processor 104 retrieves and executes the instructions. The instructions received by main memory 106 may optionally be stored on storage device 110 either before or after execution by processor 104.
  • Computer system 100 also includes a communication interface 118 coupled to bus 102. Communication interface 118 provides a two-way data communication coupling to a network link 120 that is connected to a local network 122. For example, communication interface 118 may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, communication interface 118 may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, communication interface 118 sends and receives electrical, electromagnetic or optical signals that carry digital data streams representing various types of information.
  • Network link 120 typically provides data communication through one or more networks to other data devices. For example, network link 120 may provide a connection through local network 122 to a host computer 124 or to data equipment operated by an Internet Service Provider (ISP) 126. ISP 126 in turn provides data communication services through the world wide packet data communication network now commonly referred to as the “Internet” 128. Local network 122 and Internet 128 both use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals on network link 120 and through communication interface 118, which carry the digital data to and from computer system 100, are exemplary forms of carrier waves transporting the information.
  • Computer system 100 can send messages and receive data, including program code, through the network(s), network link 120 and communication interface 118. In the Internet example, a server 130 might transmit a requested code for an application program through Internet 128, ISP 126, local network 122 and communication interface 118. The received code may be executed by processor 104 as it is received, and/or stored in storage device 110, or other non-volatile storage for later execution. In this manner, computer system 100 may obtain application code in the form of a carrier wave.
  • Each of the modes of play will now be described.
  • A. Quick Play
  • Quick play allows playing the game without going through the challenges of the Career Mode (section 0). It allows for loading song files and playing them freely. Within the quick play a player can select either Training or Dynamic mode.
  • 1. Quick Play Modes
  • After having selected a Quick Play, users are able to select between two different modes of play:
  • Training Mode
  • Training Mode allows a user to practice the game on any song that can be loaded into the game.
  • Dynamic Mode
  • Dynamic Mode is similar to a Training mode with the exception that there is a number of different dynamics that can be switched on and off. These dynamics pose an additional challenge. Each dynamic has a number of variables that can be adjusted and will gradually change on a player as the game is progressing. The following is an exemplary list of dynamics:
      • Gradual Tempo Adjustment: On|Off
      • Gradual Hit Window Closing: On|Off
      • Random Color Dropping: On|Off
      • Transform to Notation: On|Off
  • All of the above dynamic game elements are further defined under Starting Options in section 0, page 26.
  • 2. Load Song
  • Load Song is a part of the game that involves loading a song stored locally on the computer's hard drive. Users are able to browse for a specific song on their hard drive through a file search or simplified song browser.
  • 3. Internet Songs
  • Even though the invention will not be capable of loading songs directly off a Website into the game, it will feature a link that will open a web browser (e.g. Internet Explorer, or Netscape) to a section of the web site where more songs can be downloaded from. Users are able to download songs manually from a website and then open them from within the game. The link is provided on the splash screen.
  • B. Career Mode
  • Career mode allows a player to complete the game by going through different piano methods and levels, facing different challenges within levels.
  • 1. Overview
  • Within Piano Methods (section 0) we have Levels (section 0). Levels have a predefined number of Challenges specified and available (section 0). One can apply any challenge from within the level to any song from within the level. The level is completed when the level requirements are met. The level requirements can be, for instance, to complete a minimum of 7 out of 10 songs, and apply a minimum of 3 out of 5 challenges for each song played. Once the level requirements are met, the next level is unlocked. Higher level scores can be achieved by doing more than a minimum requirement. Additional effort is rewarded with more points, and unlocking of additional graphic themes.
  • 2. Profiles
  • Profiles are required to play the game in the career mode. Profiles keep track of the overall game progress in the career mode separately for each game user.
  • Create Profile
  • One of the first things a user must do, before he can start playing in the career mode, is to create a profile. Users are able to select an icon from a pool of available icons and assign it to their name, similar to the way users on Windows XP operating system can before they can start using the system. Then, the game will prompt for their name, and what mode they would like to start playing the game in, i.e. Beginner, Student, or Teacher. All the above information is saved onto the computer hard drive and kept there along with the entire game progress for that user.
  • Load Profile
  • If a user is returning to the game and has an existing profile already saved, he can choose his profile from a list of available profiles. This way one can continue playing, from where one left off last time.
  • Delete Profile
  • Users are able to delete existing profiles. A confirmation dialog will popup to make sure that profiles are not accidentally deleted.
  • 3. Piano Methods
  • Users are able to select a piano method from a list of available methods. On the piano method selection screen, there is a top players list with their scores, number of levels completed, and total time spent on that method. Each piano method available for play is listed along with an icon, and some short text description of the method.
  • 4. Levels
  • Users are able to select a level from within a piano method. Higher levels are unlocked by completing previous levels. The layout of the level selection screen are similar to piano method selection screen. Each level selection on the list is displayed with a brief description, and an icon. A top players list will also be displayed for each level highlighted for selection. Statistics of the progress accomplished on a level will also be displayed once a level is highlighted on a list. Exemplary statistics will include: total time spent playing, and a number of challenges completed so far within a level.
  • Each level has a predefined minimum number of songs and challenges that need to be completed in order to advance to the next level. Also, each level has a reward in form of a theme that can be unlocked when a level is completed.
  • 5. Songs
  • Songs are the last item in the hierarchy of Methods and Levels. A number of songs are available for each level. Songs that are available within the career mode are pre-configured, and for instance, track editing is not allowed. Playing these songs is on an as-is basis without the ability to alter the game environment, like it is possible in a Quick Play mode described in section 0.
  • Songs that are available within piano methods can be in any file format that is supported by the invention. Songs available from within piano methods are irreplaceable from the operating system. In other words, users will not be able to replace these songs with ones that are less difficult, in order to score better in the game.
  • 6. Challenges
  • The Game features a number of challenges for its users. The challenges are designed to assist in training in a variety of techniques. This invention will feature the following types of challenges:
  • Tempo
  • Starts at a 50% of the song file tempo and gradually increases. Bonus points are awarded at playing faster than default song file tempo. Players need to stay within the accuracy parameter to continue playing, i.e. 90%. Maximum allowed is a parameter that can be defined by the creator of the challenge.
  • Rhythmic Precision Challenge
  • In this challenge the hit window gradually closes to a challenge creator's predefined size. The starting size is a user definable parameter. Players need to stay within the accuracy parameter to continue playing.
  • Primary Beats
  • The challenge is to stay within a predefined accuracy level hitting primary beats. Primary beats are defined by the challenge creator. There is no penalty for hitting other (than primary beats) notes within the hit window.
  • Off-Beats
  • Similarly to Primary Beats challenge, Off-Beats Challenge requires to stay within a predefined accuracy level, hitting off-beats. There is no penalty for hitting other (than off-beats) notes within the hit window.
  • Color Drop Out
  • At selected or random times game colored objects representing notes are turned into black & white. The amount of game objects that lose color gradually increases as a song is progressing.
  • Note Duration (Legato)
  • The challenge is to hold down the piano keys for the correct duration of notes. Players need to stay within the accuracy parameter to continue playing, say 90%.
  • Rotation
  • The screen rotates from vertical to horizontal position. Players need to stay within the accuracy parameter to continue playing. As users master this challenge, they will learn the relationships between the keyboard and standard musical rotation.
  • Visibility
  • A series of visibility challenges are introduced to induce the player to memorize the piece of music that he/she is paying. For instance, a cloud could enter the game field and cover parts of the screen, making it difficult for a user to see the game objects. Visibility challenges are further defined during the development.
  • Loudness (Velocity)
  • Certain sections of a song will have different velocity parameters (forte, piano, etc.) that users need to stay within. Players need to stay within the accuracy parameter to continue playing. This challenge will only be available on touch-sensitive piano keyboards.
  • 7. Creating Custom Piano Methods
  • Creating custom piano methods involves creating an empty method, naming it, creating levels within, and adding songs to levels. Custom piano methods do not include rewards, high-score lists or graphical themes. All challenges can be played on any song that is part of the custom piano method. Custom piano methods should be savable in a manner so that they can be copied and shared among users of the invention. Custom Piano methods are folders with files, and simple configuration files. All levels are unlocked and playable in custom piano methods. The idea of custom piano methods is to allow teachers for creating assignments for their students that are users of the invention.
  • C. Starting Options
  • Starting Options are options that can be configured before a user starts playing the game. The number of available starting options depends on the game mode. In some modes some options are not available, and are grayed out, or left out. However, regardless of what mode the game was started in, in order to start playing, one will have to go through these options. One exception to this rule is an Instant Play which omits configuring the Starting Options, and assumes all default values for playing. Preferred availability of Starting Options in different game modes is included in the CD-ROM Appendix.
  • The following is a list of all starting options along with brief descriptions:
  • Screen Orientation: V|H|Rotation
  • This option specifies the orientation of the screen. Some users may prefer to start the game play in a Vertical mode, whereas some will prefer the Horizontal mode. It is also possible to start the game play in a mode where the screen will gradually start rotating from Vertical to Horizontal orientation.
  • Annotation: On|Off
  • This switch turns annotations on/off. Annotations are included in the .amm internal file format and involve displaying a kind of “post-it” text notes on the game screen as annotated game objects scroll.
  • Fingerings: On|Off
  • This switch turns fingering information on/off. Similarly to annotations, fingerings are included in the .amm internal file format. When fingerings are enabled, game objects, if marked so, will display a number from 1 to 5 serving as a guide to a user as to what finger to use in order to target the game object correctly on the piano keyboard.
  • Freeze: On|Off
  • This switch enables or disables the ability to access a Freeze Menu from within the Game Play, as discussed hereinafter.
  • Beats Per Screen
  • This option allows the user to select how many beats or measures are on the screen vertically during game play, allowing fast songs enough anticipation to prepare for the next note.
  • Objects: Long|Short|Notation
  • This option allows for selection between different types of game objects. Long objects are those that can be drawn to represent the actual durations of musical notes, for instance: caterpillars. Whereas, short objects are those that only represent the starting point of a musical note, and do not represent the duration, i.e. Soccer Balls or Balloons. When Notation is selected, instead of game objects a user will see musical notation during the Game Play.
  • Notation: Accidentals|Key Signature
  • This option allows for selection between different types of musical notation. It is only available if Notation is selected as a game object.
  • Theme Selection
  • Theme selection is a set of GUI elements that allow for selection of different game themes. Themes include backgrounds and different game objects. Different backgrounds can be combined with different game objects. However, themes that were not yet unlocked in the Career Mode (section 0) are not available for selection.
  • Color: On|Off
  • This switch turns on/off the color of the game objects as well as the color of the virtual piano keyboard on the screen. Basically, if toggled ON—all game objects and piano keys are displayed color-coded, otherwise, these objects are black & white.
  • Track(s) Select (incl. Range Preview, Active Keys)
  • This section allows for a visual selection of the tracks in a loaded song file. Once a song is loaded, a list of available tracks are displayed, and a user will have the ability to select those tracks that he/she wishes to play in the game. Once a track is selected by a user, the program will automatically find the range of musical notes for it, and mark it on the piano keyboard displayed on the screen (i.e. C2-B3). The range will always be calculated for all selected tracks. For instance, if two tracks are selected: track1 and track2—the range is marked for both as if these were one track. Previewing the range is important because it allows a user to see if range of selected tracks fits onto the attached MIDI keyboard. i.e. playing a piece that spreads across 6 octaves on a keyboard that only has 4 would be virtually impossible. Additionally users are able to click on individual piano keys and mark them as active or inactive. By default all keys in the range are marked active. During the Game Play user assumes responsibility for hitting only those piano keys that are active.
  • Instrument Select for Track(s)
  • Users are given the ability to change General MIDI instruments for any of the selected tracks that are to be played in the game. Users can simply select one instrument per track from a drop down menu of 128 General MIDI Level 1 Instruments.
  • Accompaniment Track(s) Select
  • Users are also given the ability to select accompaniment tracks for the songs that they wish to play for the game. Accompaniment tracks are played audibly in the background. Some will find accompaniment helpful, while others distracting, therefore these selections are optional.
  • Instrument Select for Accompaniment Track(s)
  • Similarly to selecting instruments for the tracks to be played in the game, users are given the ability to change General MIDI instruments for any of the selected accompaniment tracks. Users can simply select one instrument per accompaniment track from a drop down menu of 128 General MIDI Level 1 Instruments.
  • Merging Selected Tracks
  • This option allows merging together a number of selected tracks into one. This operation does not alter the content of the original file, instead it is performed in the computer's memory on the file loaded.
  • Delete Track(s)
  • Deletes selected track(s) from memory. Deleted track(s) are not deleted from a song file.
  • Split Hands for a Track
  • If only one track is selected, this option is enabled. Executing this option splits a single track into two tracks with musical content arranged separately for both hands. I.e. The first track holds musical notes to be played with a left hand, and the second track is to be played with a right hand.
  • Transpose Selected Track(s)
  • This option transposes selected track(s) by semitones either up or down.
  • Track Setup
  • This section allows a user to configure MIDI output ports and channels for any track of the file loaded. For instance, a user might prefer to hear an accompaniment track #2 on the 11th channel of a synthesizer located on the 3rd MIDI port.
  • Metronome: On|Off
  • This switch enables or disables the audible metronome heard during the game play.
  • Practice Challenge Selection
  • This section allows for selecting a challenge from all the challenges that are built into the game, and are part of the Career Mode, as discussed earlier. By allowing this, users can flexibly practice different challenges on any song they wish to.
  • Tempo Select
  • This option allows for a selection of a tempo that is used during the game play. Part of this option is a switch called “MIDI tempo: On|Off” that enables playing at a tempo specified in the currently loaded song. If this switch is enabled and a current song file features variations in tempo, the game speed will adjust automatically.
  • Hit Window Adjustment
  • This option allows for adjusting the height of the hit window of the game.
  • Background: On|Off|Opacity %
  • An option that enables, disables or allows for an adjustment of opacity of the background. Opacity values range from 0% to 100%.
  • Measure: On|Off
  • This switch enables or disables the visual measurement that scrolls along with the game objects.
  • Object Guides: On|Off This switch enables of disables visible trajectories guide lines.
  • Stats (Overlays): On|Off
  • This switch turns on or off on-screen game statistics, score, etc.
  • Keyboard: Auto|2 oct.|4 oct.|5 oct.
  • This option allows for selecting the size of the on-screen keyboard used in the game play. Users can select between 2-octave, 4-octave or 5-octave versions of keyboards. Automatic keyboard size selection is also available. The game will support three keyboard sizes: 2-, 4-, and 5-octave.
  • Starting Key Selection
  • This option allows for selecting the starting key on the on-screen keyboard. This is very helpful in calibrating the physical MIDI keyboard with the on-screen keyboard if the two are of a different size. The selections are: C0, C1, C2 . . . C11.
  • Gradual Tempo Adjustment: On|Off
  • Enabling this option will gradually increase or decrease the tempo of the game play to the tempo specified in a loaded song file. For instance, by starting the game play at a low tempo, users can practice a piece of music at a slowly, but constantly increasing tempo.
  • Gradual Hit Window Closing: On|Off
  • Enabling this option will gradually decrease the height of the hit window during the game play. For instance, by starting the game play at a hit window wide open, users can practice a piece of music with a slowly, but constantly increasing precision.
  • Random Color Dropping: On|Off
  • If this option is enabled, during the game play, game objects will start losing color selectively or at random, making it harder to color-coordinate game objects with the color-coded piano keyboard. Both game objects and the keys on the on-screen keyboard can drop colors.
  • Transform to Notation: On|Off
  • Enabling this option will gradually transform game objects into musical notation.
  • Loop Song: On|Off
  • Enables or disables looping of a song in the game play.
  • Lyrics: On|Off
  • If a song file includes lyrics and this option is enabled, lyrics will appear on the screen.
  • Auto Save Song: On|Off
  • This option enables or disables auto-saving of all alternations to a currently loaded song. If auto save song is enabled, a user is prompted for a new file name for a song. Songs are saved into an .amm internal song file format.
  • SFX Volume & Music Volume
  • These options are implemented in form of sliders and allow for adjusting volume levels of both, sound effects and the music.
  • Extract Part
  • This is a hybrid of a track merge option. A user chooses the clefs (tracks) to be merged.
  • Prepare Sing Along Part
  • This option is merely separating out the text into measures, for someone to sing along without the notation being on the same sheet.
  • Prepare Fake Book Lead Sheet
  • “Fake” books are collections of popular songs with the lyrics and the chord names (Gm7) for jazz players, and maybe the lead melody, and they would improvise the rhythm and harmonic parts, using the melody and chord structure as a skeleton. MidiNotate software extracts the lyrics, lead melody, and chord names from a midi file, and allows for printing it in this manner.
  • Transcribe Chord Names
  • Chord names have multiple ways of being described. An example might be A Major. It could be written simply A or it might be written as A maj.
  • All of the above options can be combined into groups. For instance there is a large amount of Boolean options (two state, on|off) that are implemented as check boxes in the GUI of the game.
  • D. Game Play
  • This is where all the action happens. Game play is the actual game.
  • 1. Keyboard Shortcuts
  • There is a number of exemplary keyboard shortcuts linked to different options, which can be invoked during the game play without stopping the flow of the game. Other options are possible. These options are:
  • Tempo Adjustment Arrow Left | Arrow Right
    Hit Window Adjustment Arrow Up | Arrow Down
    Warning Line Adjustment Shift + Arrow Up | Shift 4 + Arrow Down
    Overall Volume Adjustment Gray + | Gray −
    Metronome M
  • Some adjustments are not available in some challenges, e.g. tempo adjustment is not available during the game play in tempo challenge.
  • 2. Missed Notes
  • Whenever a player misses a note, a mark is placed onto the game field. Users are able to scroll up and down the entire song from within the freeze menu and see where they have missed.
  • 3. Musical Notation
  • The invention is capable of transforming MIDI files into a musical notation to be displayed on the screen during the game play. Examples of software suitable for implementing this function can be found at:
  • Notation Technologies http://www.notationtechnologies.com/
    Notation Software, Inc. http://www.notation.com/
  • 4. Game Objects
  • The invention features a number of game objects, also known as Sprites. All game objects will feature 32-bit color and alpha-blending support. The game objects are two dimensional, but drawn on the screen using 3D techniques in order to take advantage of hardware accelerated 3D drawing. This will allow for an easy scaling, rotating and alpha-blending.
  • 5. On-Screen Piano Keyboard
  • The on-screen piano keyboard comes in three different sizes: 2-, 4-, and 5-octave. Depending on the keyboard size selected, size of the game object will change accordingly to keep the width proportional to the width of the on-screen keyboard keys.
  • 6. Point Scoring
  • The game rewards users with points for correctly hit musical notes, playing with different dynamic options enabled, and for playing different challenges, and so on.
  • E. Freeze Menu
  • Freeze Menu can be invoked from within the game play only. It will allow for alternation of some of the game parameters, making annotations on the screen and saving a currently played song into an internal file format along with all annotations, highlighting and fingering information.
  • 1. Invoking
  • The freeze menu is invoked by pressing an ESC key on the keyboard from within the game play. Once ESC is pressed the following happens: the entire game freezes, vertical scrollbar appears on the screen to allow scrolling through the entire song both, back to the parts already played, and ahead. Note that scrolling, annotating, highlighting or fingering is not available when the game screen is in the middle of a rotation. Once the game is frozen, a translucent help box is displayed with directions as to what can be done on the screen. For instance, a typical text would read: “To create an annotation on the game screen, click on an empty space”.
  • 2. Highlighting and Fingering When a user clicks on any game object, a pop-up menu, similar to the one illustrated below, will appear next the object:
  • Figure US20100242709A1-20100930-C00001
  • From within that pop-up menu users can select the highlight color or a fingering number for either of hands, just by clicking on squares (as illustrated above). Whatever selection was made on the pop-up menu, it would be applied to the object that was clicked on. Highlighting should be done in form of a halo effect around the object. Colors illustrated on the pop-up menu above, are for illustration purposes only. Additionally, if an object has already a fingering or a highlighting assigned, those can be removed by clicking on “x” buttons as illustrated above.
  • 3. Annotating
  • Users are able to place notes on the game screen. Those annotations later become a part of the .amm file. Annotations once placed on the game screen are remembered in the file if desired (save). Users can add and remove annotations. Annotations are placed on the screen in a form of speech balloons. There are a number of balloon shapes to choose from. In order to add an annotation to the game screen, a user would click on an empty space on the screen. Once clicked, a popup menu will appear allowing for selecting the shape of the speech balloon and the direction of its arrow-pointer. The text would be entered then through a text edit box. Annotations can be removed from the game screen. In order to remove an annotation one would have to click on an already existing annotation, and then click on an “x” button that would appear next to it. The game will also allow for editing already entered annotations. In order to edit an annotation, similarly to deleting one, a user would have to click on an already placed annotation and then click on an “edit” button that would appear next to it. Editing annotations are done in the same way as entering them.
  • 4. Freeze Menu Options
  • The number of options in the freeze menu is dependent on the game mode. For a preferred availability of freeze menu options in different game modes please see Attachment B. Freeze menu options are listed below. It is worth noting that a significant number of these options are a subset of Starting Options, which have been defined in section C above. Definitions of already defined options are omitted here.
  • Fingerings: On|Off
  • Objects: Long|Short|Notation
  • Notation: Accidentals|Key Signature
  • Theme Selection
  • Color: On|Off
  • Track(s) Select (incl. Range Preview, Active Keys)
  • Instrument Select for Track(s)
  • Accompaniment Track(s) Select
  • Instrument Select for Accompaniment Track(s)
  • Transpose Selected Track(s)
  • Metronome: On|Off
  • Tempo Select
  • Hit Window Adjustment
  • Background: On|Off|Opacity %
  • Measure: On|Off
  • Object Guides: On|Off
  • Stats (Overlays): On|Off
  • Starting Key Selection
  • Gradual Tempo Adjustment: On|Off
  • Gradual Hit Window Closing: On|Off
  • Random Color Dropping: On|Off
  • Transform to Notation: On|Off
  • Loop Song: On|Off
  • Lyrics: On|Off
  • SFX Volume & Music Volume
  • Auto Save Song: On|Off
  • Turns On or Off auto saving song feature. If this option is enabled, songs are automatically saved into a previously specified file as changes are made.
  • Save Song As . . .
  • Prompts for a file name; saves a current song into an .amm file.
  • F. Game Setup
  • Game setup can be invoked from the main menu. The following are its options:
  • Menu Options: Advanced|Beginner
  • This menu option allows selecting between two different sets of menu options in the game. Advanced set is designed for users who want to take the game to the fullest and require maximum flexibility, whereas Beginner set eliminates a number of menu options that are simply too complex for an average user, or a child.
  • Gamma Correction
  • This option allows adjusting the gamma value of the screen.
  • Midi in Select
  • This menu option allows for selection of the MIDI input port for the game. The game will enumerate all possible MIDI ports on the system and make them available for selection.
  • Midi Out Select
  • This menu option allows for selection of the MIDI output port for the game. The game will enumerate all possible MIDI ports on the system and make them available for selection.
  • Game Detail Level: Small|Medium|High
  • Game detail level enables and disables some of the CPU intensive game visual effects. Small level of detail will make it possible to run the game on a low-end computer at the price of visual effects and overall eye-candy.
  • Music & SFX Volume Level
  • These menu options in form of sliders adjust the overall sound mixer values for the game. SFX Volume option allows for an adjustment of the overall volume level of all .wav sounds played in the game (PCM channels of the sound card), whereas Music Volume allows for an adjustment of the overall volume level of the internal sound card synthesizer. Both sliders can be assigned values from 0% to 100%.
  • Keyboard Range Configuration
  • This is an option that will determine the size of the keyboard attached to the computer. A user is asked to press the very left and the very right key on the MIDI keyboard, and the game will calculate the octave size of the keyboard.
  • Velocity Test
  • This option will allow one to test a user's MIDI keyboard to determine whether it is velocity sensitive or not. A user is asked to strike a key on the MIDI keyboard twice, first gently, and hard the second time. Based on the difference in value of the received MIDI velocity message, the program will determine if a user has a touch-sensitive keyboard.
  • Although the present invention has been described and illustrated in detail, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation, the spirit and scope of the present invention being limited only by the terms of the appended claims.
  • APPENDIX A CD-ROM Contents
  • The following is a description of the contents of a CD-ROM submitted as an appendix to this application:
  • The CD-ROM contains a plurality of directories and files as shown in the following printout of directories from the CD-ROM.
  • The file extension description of most files adequately specifies to one skilled in the are how to open them. The files can be opened using Microsoft Word, Microsoft Excel, or an adobe Acrobat Reader. The .vsd file can be read using Visio.
  • Software for opening any other files that may not be self evident can be found at http://extsearch.com or similar file extension directory search engines.

Claims (20)

1. Apparatus for providing musical instruction comprising:
a. a computing element;
b. a display connected to the computing element;
c. a musical keyboard connected to the computing element; and
d. software for execution on said computing element linking a virtual keyboard displayed on said display when the software is executed on the computing element with actions by a user of said musical keyboard as part of a game.
2. Apparatus of claim 1 in which the software translates notes of a representation of a musical piece into game objects on said display.
3. Apparatus of claim 2 in which the software comprises an interpreter for converting a representation of a musical piece into commands for generating game objects.
4. Apparatus of claim 3 in which commands for generating game objects drive a game object generator.
5. Apparatus of claim 4 in which the game object generator is associated with a library of game objects.
6. Apparatus of claim 4 in which the game object generator is associated with a set of parameters for setting properties of game objects.
7. Apparatus of claim 2 in which the game objects move with time from an initial launch position on the display toward keys of the virtual keyboard which correspond to musical notes of the musical piece.
8. Apparatus of claim 7 in which game objects move along pre-defined trajectories.
9. A method of using a computer comprising the step of:
linking a musical keyboard connected to the computer with the action of game objects displayed on a display of the computer with respect to a virtual keyboard displayed on said display.
10. The method of claim 9 in which game objects are generated for notes of a musical piece stored in a computer readable format.
11. The method of claim 10 in which the musical piece of one of a library of musical pieces stored on said computer.
12. The method of claim 10 in which the computer readable format is MIDI.
13. The method of claim 9 in which a user receives a reward in the form of at least one of an audio playback, a video or graphical animation display or an award of points.
14. The method of claim 13 in which the reward is granted when a user depresses a key of the musical keyboard within a time window spanning an instant when a game object intersects with the corresponding key of a virtual keyboard.
15. The method of claim 14 in which at least one profile of user information is stored on said computer for each of a plurality of users.
16. The method of claim 14 in which the profile information may include a set of starting options.
17. The method of claim 16 in which the set of starting options may be selected from the group consisting of: screen orientation mode, annotation on/off, fingering information on/off, freeze frame on/off; objects, notation, theme, color, tracks, instruments for tracks, accompanyment, tracks to be merged, tracks to be deleted, split hands for tracks, transpose selected tracks, output port, metronome on/off, practice challenge selection, tempo select, background, measure, object guides on/off, statistics overlay, keyboard size, number of beats or measures on screen at once, starting key, gradual tempo adjustment, gradual hit window closing, random color dropping, transform object into musical notation, loop song, lyrics on/off, autosave song on/off; and volume.
18. The method of claim 14 in which a plurality of methods are available for selection by a user, each method comprising at least one song and a plurality of levels of difficulty for each song.
19. The method of claim 18 in which each level of difficulty comprises a plurality of songs and one or more challenges designed to advance skills in a respective area of competence.
20. The method of claim 19 in which the challenges are selected from the group consisting of: a tempo challenge; a rhythmic precision challenge, a primary beats challenge, an off-beats challenge, a color drop out challenge, a note duration challenge, a rotation challenge, a visibility challenge, and a loudness challenge.
US12/789,271 2001-10-20 2010-05-27 Interactive game providing instruction in musical notation and in learning an instrument Abandoned US20100242709A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US12/789,271 US20100242709A1 (en) 2001-10-20 2010-05-27 Interactive game providing instruction in musical notation and in learning an instrument

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US34755401P 2001-10-20 2001-10-20
US10/273,353 US7174510B2 (en) 2001-10-20 2002-10-18 Interactive game providing instruction in musical notation and in learning an instrument
US11/411,835 US7739595B2 (en) 2001-10-20 2006-04-27 Interactive game providing instruction in musical notation and in learning an instrument
US12/789,271 US20100242709A1 (en) 2001-10-20 2010-05-27 Interactive game providing instruction in musical notation and in learning an instrument

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US11/411,835 Continuation US7739595B2 (en) 2001-10-20 2006-04-27 Interactive game providing instruction in musical notation and in learning an instrument

Publications (1)

Publication Number Publication Date
US20100242709A1 true US20100242709A1 (en) 2010-09-30

Family

ID=32174517

Family Applications (4)

Application Number Title Priority Date Filing Date
US10/273,353 Expired - Fee Related US7174510B2 (en) 2001-10-20 2002-10-18 Interactive game providing instruction in musical notation and in learning an instrument
US10/531,642 Expired - Fee Related US7799984B2 (en) 2002-10-18 2003-05-30 Game for playing and reading musical notation
US11/411,835 Expired - Fee Related US7739595B2 (en) 2001-10-20 2006-04-27 Interactive game providing instruction in musical notation and in learning an instrument
US12/789,271 Abandoned US20100242709A1 (en) 2001-10-20 2010-05-27 Interactive game providing instruction in musical notation and in learning an instrument

Family Applications Before (3)

Application Number Title Priority Date Filing Date
US10/273,353 Expired - Fee Related US7174510B2 (en) 2001-10-20 2002-10-18 Interactive game providing instruction in musical notation and in learning an instrument
US10/531,642 Expired - Fee Related US7799984B2 (en) 2002-10-18 2003-05-30 Game for playing and reading musical notation
US11/411,835 Expired - Fee Related US7739595B2 (en) 2001-10-20 2006-04-27 Interactive game providing instruction in musical notation and in learning an instrument

Country Status (3)

Country Link
US (4) US7174510B2 (en)
AU (1) AU2003301598A1 (en)
WO (1) WO2004038690A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110239845A1 (en) * 2010-03-31 2011-10-06 Yamaha Corporation Musical score display apparatus and program for realizing musical score display method
US20120090446A1 (en) * 2010-04-20 2012-04-19 Sylvain Jean-Pierre Daniel Moreno System and method for providing music based cognitive skills development
US20120192700A1 (en) * 2009-06-01 2012-08-02 Starplayit Pty Ltd Music display system
US8642871B2 (en) 2008-11-24 2014-02-04 Piano Matchmaker Llc Instructional music reading and instrument playing system and method
US20190228754A1 (en) * 2018-01-23 2019-07-25 Roland VS LLC Generation and transmission of musical performance data
US11386803B1 (en) 2010-04-20 2022-07-12 Sylvain Jean-Pierre Daniel Moreno Cognitive training system and method

Families Citing this family (128)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9852649B2 (en) * 2001-12-13 2017-12-26 Mind Research Institute Method and system for teaching vocabulary
JP3578273B2 (en) * 2002-02-22 2004-10-20 コナミ株式会社 General-purpose keyboard setting program for keyboard game program
US7629527B2 (en) * 2002-03-11 2009-12-08 Hellene Hiner Machine and method for teaching music and piano
US20030214539A1 (en) * 2002-05-14 2003-11-20 Microsoft Corp. Method and apparatus for hollow selection feedback
US8491311B2 (en) * 2002-09-30 2013-07-23 Mind Research Institute System and method for analysis and feedback of student performance
US20040137984A1 (en) * 2003-01-09 2004-07-15 Salter Hal C. Interactive gamepad device and game providing means of learning musical pieces and songs
US7435891B2 (en) * 2003-05-30 2008-10-14 Perla James C Method and system for generating musical variations directed to particular skill-levels
US9778919B2 (en) * 2003-06-27 2017-10-03 Adobe Systems Incorporated Dual context interaction with a content object for facilitating content creation and software development
US7742832B1 (en) * 2004-01-09 2010-06-22 Neosonik Method and apparatus for wireless digital audio playback for player piano applications
US7288712B2 (en) * 2004-01-09 2007-10-30 Yamaha Corporation Music station for producing visual images synchronously with music data codes
JP3847753B2 (en) * 2004-01-30 2006-11-22 株式会社ソニー・コンピュータエンタテインメント Image processing apparatus, image processing method, recording medium, computer program, semiconductor device
US7806759B2 (en) * 2004-05-14 2010-10-05 Konami Digital Entertainment, Inc. In-game interface with performance feedback
KR100456487B1 (en) * 2004-08-02 2004-11-09 엔에이치엔(주) personal icon providing system and method thereof
US20070046678A1 (en) * 2005-09-01 2007-03-01 Peterson Matthew R System and method for training with a virtual apparatus
US7071404B1 (en) 2005-09-27 2006-07-04 Smith Laura A Laser activated synthesizer system
US7462772B2 (en) * 2006-01-13 2008-12-09 Salter Hal C Music composition system and method
US20070163428A1 (en) * 2006-01-13 2007-07-19 Salter Hal C System and method for network communication of music data
US7538266B2 (en) * 2006-03-27 2009-05-26 Yamaha Corporation Electronic musical apparatus for training in timing correctly
US7459624B2 (en) 2006-03-29 2008-12-02 Harmonix Music Systems, Inc. Game controller simulating a musical instrument
WO2007124387A2 (en) * 2006-04-19 2007-11-01 Allegro Multimedia, Inc. System and method of instructing musical literacy and performance of a stringed instrument
US7538265B2 (en) * 2006-07-12 2009-05-26 Master Key, Llc Apparatus and method for visualizing music and other sounds
JP4108719B2 (en) * 2006-08-30 2008-06-25 株式会社バンダイナムコゲームス PROGRAM, INFORMATION STORAGE MEDIUM, AND GAME DEVICE
JP4137148B2 (en) * 2006-08-30 2008-08-20 株式会社バンダイナムコゲームス PROGRAM, INFORMATION STORAGE MEDIUM, AND GAME DEVICE
US8079907B2 (en) * 2006-11-15 2011-12-20 Harmonix Music Systems, Inc. Method and apparatus for facilitating group musical interaction over a network
JP5351380B2 (en) * 2006-11-16 2013-11-27 ローランド株式会社 Performance information processing apparatus and electronic musical instrument
US7576280B2 (en) * 2006-11-20 2009-08-18 Lauffer James G Expressing music
EP2092511A1 (en) * 2006-12-12 2009-08-26 Koninklijke Philips Electronics N.V. Musical composition system and method of controlling a generation of a musical composition
US20100095828A1 (en) * 2006-12-13 2010-04-22 Web Ed. Development Pty., Ltd. Electronic System, Methods and Apparatus for Teaching and Examining Music
NZ577402A (en) * 2006-12-13 2012-01-12 Web Ed Dev Pty Ltd Electronic system, methods and apparatus for teaching and examining music theory
US7767895B2 (en) * 2006-12-15 2010-08-03 Johnston James S Music notation system
JP4281798B2 (en) * 2006-12-22 2009-06-17 ソニー株式会社 Content playback system, electronic device, playback control method, and program
US8907193B2 (en) 2007-02-20 2014-12-09 Ubisoft Entertainment Instrument game system and method
US20080200224A1 (en) 2007-02-20 2008-08-21 Gametank Inc. Instrument Game System and Method
US7589269B2 (en) * 2007-04-03 2009-09-15 Master Key, Llc Device and method for visualizing musical rhythmic structures
WO2008130697A1 (en) * 2007-04-19 2008-10-30 Master Key, Llc Method and apparatus for editing and mixing sound recordings
WO2008130665A1 (en) 2007-04-19 2008-10-30 Master Key, Llc System and method for audio equalization
US7777117B2 (en) * 2007-04-19 2010-08-17 Hal Christopher Salter System and method of instructing musical notation for a stringed instrument
US7960637B2 (en) 2007-04-20 2011-06-14 Master Key, Llc Archiving of environmental sounds using visualization components
US7947888B2 (en) * 2007-04-20 2011-05-24 Master Key, Llc Method and apparatus for computer-generated music
US8073701B2 (en) * 2007-04-20 2011-12-06 Master Key, Llc Method and apparatus for identity verification using visual representation of a spoken word
WO2008130696A1 (en) * 2007-04-20 2008-10-30 Master Key, Llc Calibration of transmission system using tonal visualization components
WO2008130663A1 (en) * 2007-04-20 2008-10-30 Master Key, Llc System and method for foreign language processing
US7932455B2 (en) * 2007-04-20 2011-04-26 Master Key, Llc Method and apparatus for comparing musical works
WO2008130666A2 (en) * 2007-04-20 2008-10-30 Master Key, Llc System and method for music composition
US7453036B1 (en) 2007-06-05 2008-11-18 Beverly Henkel System and method for teaching piano
US8678896B2 (en) 2007-06-14 2014-03-25 Harmonix Music Systems, Inc. Systems and methods for asynchronous band interaction in a rhythm action game
US20090075711A1 (en) * 2007-06-14 2009-03-19 Eric Brosius Systems and methods for providing a vocal experience for a player of a rhythm action game
JP5358071B2 (en) * 2007-07-04 2013-12-04 任天堂株式会社 GAME PROGRAM AND GAME DEVICE
US9159325B2 (en) * 2007-12-31 2015-10-13 Adobe Systems Incorporated Pitch shifting frequencies
US7915514B1 (en) * 2008-01-17 2011-03-29 Fable Sounds, LLC Advanced MIDI and audio processing system and method
AU2009206663A1 (en) * 2008-01-24 2009-07-30 745 Llc Method and apparatus for stringed controllers and/or instruments
ES2324268B2 (en) * 2008-01-24 2011-02-22 Jaime Iglesias Alvaro-Gracia MUSICAL CONTROLLER.
US20090231281A1 (en) * 2008-03-11 2009-09-17 Microsoft Corporation Multi-touch virtual keyboard
US8317614B2 (en) * 2008-04-15 2012-11-27 Activision Publishing, Inc. System and method for playing a music video game with a drum system game controller
WO2009135964A1 (en) 2008-05-09 2009-11-12 Virtual Toys S.L. Device and method for emulating a virtual musical instrument
CN101598966A (en) * 2008-06-06 2009-12-09 深圳富泰宏精密工业有限公司 The contactor control device and the control method thereof that show two pictures
US8294015B2 (en) * 2008-06-20 2012-10-23 Randy Lawrence Canis Method and system for utilizing a gaming instrument controller
US20090328158A1 (en) * 2008-06-30 2009-12-31 Nokia Corporation Method, apparatus, and computer program product for providing software application invitation
US8663013B2 (en) 2008-07-08 2014-03-04 Harmonix Music Systems, Inc. Systems and methods for simulating a rock band experience
US8858330B2 (en) * 2008-07-14 2014-10-14 Activision Publishing, Inc. Music video game with virtual drums
WO2010059994A2 (en) 2008-11-21 2010-05-27 Poptank Studios, Inc. Interactive guitar game designed for learning to play the guitar
US7842872B2 (en) * 2008-12-30 2010-11-30 Pangenuity, LLC Steel pan tablature system and associated methods
US8158869B2 (en) * 2008-12-30 2012-04-17 Pangenuity, LLC Music teaching tool for steel pan and drum players and associated methods
US20100184497A1 (en) * 2009-01-21 2010-07-22 Bruce Cichowlas Interactive musical instrument game
US8237042B2 (en) * 2009-02-18 2012-08-07 Spoonjack, Llc Electronic musical instruments
US8362347B1 (en) * 2009-04-08 2013-01-29 Spoonjack, Llc System and methods for guiding user interactions with musical instruments
US8445767B2 (en) * 2009-04-11 2013-05-21 Thomas E. Brow Method and system for interactive musical game
US20100293365A1 (en) * 2009-05-14 2010-11-18 Sony Computer Entertainment America Inc. Method and apparatus for system and software quick launch
US8465366B2 (en) 2009-05-29 2013-06-18 Harmonix Music Systems, Inc. Biasing a musical performance input to a part
US8017854B2 (en) 2009-05-29 2011-09-13 Harmonix Music Systems, Inc. Dynamic musical part determination
US8449360B2 (en) * 2009-05-29 2013-05-28 Harmonix Music Systems, Inc. Displaying song lyrics and vocal cues
US8076564B2 (en) * 2009-05-29 2011-12-13 Harmonix Music Systems, Inc. Scoring a musical performance after a period of ambiguity
US20100304810A1 (en) * 2009-05-29 2010-12-02 Harmonix Music Systems, Inc. Displaying A Harmonically Relevant Pitch Guide
US7923620B2 (en) * 2009-05-29 2011-04-12 Harmonix Music Systems, Inc. Practice mode for multiple musical parts
US8026435B2 (en) * 2009-05-29 2011-09-27 Harmonix Music Systems, Inc. Selectively displaying song lyrics
US7935880B2 (en) 2009-05-29 2011-05-03 Harmonix Music Systems, Inc. Dynamically displaying a pitch range
US20100304811A1 (en) * 2009-05-29 2010-12-02 Harmonix Music Systems, Inc. Scoring a Musical Performance Involving Multiple Parts
US7982114B2 (en) * 2009-05-29 2011-07-19 Harmonix Music Systems, Inc. Displaying an input at multiple octaves
US8080722B2 (en) * 2009-05-29 2011-12-20 Harmonix Music Systems, Inc. Preventing an unintentional deploy of a bonus in a video game
US9981193B2 (en) 2009-10-27 2018-05-29 Harmonix Music Systems, Inc. Movement based recognition and evaluation
WO2011056657A2 (en) 2009-10-27 2011-05-12 Harmonix Music Systems, Inc. Gesture-based user interface
US8636572B2 (en) 2010-03-16 2014-01-28 Harmonix Music Systems, Inc. Simulating musical instruments
US20110281639A1 (en) * 2010-04-07 2011-11-17 Tucoola Ltd. Method and system of monitoring and enhancing development progress of players
US8562403B2 (en) 2010-06-11 2013-10-22 Harmonix Music Systems, Inc. Prompting a player of a dance game
EP2579955B1 (en) 2010-06-11 2020-07-08 Harmonix Music Systems, Inc. Dance game and tutorial
US9358456B1 (en) 2010-06-11 2016-06-07 Harmonix Music Systems, Inc. Dance competition game
US8119896B1 (en) 2010-06-30 2012-02-21 Smith L Gabriel Media system and method of progressive musical instruction
US8664503B2 (en) 2010-08-13 2014-03-04 Antakamatics, Inc. Musical notation and method of teaching same
US8822801B2 (en) * 2010-08-20 2014-09-02 Gianni Alexander Spata Musical instructional player
US9024166B2 (en) 2010-09-09 2015-05-05 Harmonix Music Systems, Inc. Preventing subtractive track separation
US8591339B2 (en) 2011-06-29 2013-11-26 Sony Computer Entertainment America Llc Method and apparatus for representing computer game player information in a machine-readable image
US9082380B1 (en) * 2011-10-31 2015-07-14 Smule, Inc. Synthetic musical instrument with performance-and/or skill-adaptive score tempo
US8987572B2 (en) * 2011-12-29 2015-03-24 Generategy Llc System and method for teaching and testing musical pitch
US9269276B2 (en) * 2012-02-17 2016-02-23 Baby Grand Products Llc Music teaching aid
US8389848B1 (en) * 2012-02-20 2013-03-05 BakBeat, LLC Electro-mechanically assisted bass drum pedal and method of use
JP2013182195A (en) * 2012-03-02 2013-09-12 Casio Comput Co Ltd Musical performance device and program
JP6127367B2 (en) 2012-03-14 2017-05-17 カシオ計算機株式会社 Performance device and program
JP6024136B2 (en) * 2012-03-15 2016-11-09 カシオ計算機株式会社 Performance device, performance method and program
US8686268B1 (en) 2012-03-16 2014-04-01 Natalie Young System of associating sheet music notation with keyboard keys and sight reading
US8768751B2 (en) * 2012-04-25 2014-07-01 Sap Ag Enterprise gamification system for awarding employee performance
CN103943098A (en) * 2013-01-17 2014-07-23 李宋 DoMiSo symphony instrument
WO2014137311A1 (en) 2013-03-04 2014-09-12 Empire Technology Development Llc Virtual instrument playing scheme
US9409092B2 (en) 2013-08-03 2016-08-09 Gamesys Ltd. Systems and methods for integrating musical features into a game
KR20150072597A (en) 2013-12-20 2015-06-30 삼성전자주식회사 Multimedia apparatus, Method for composition of music, and Method for correction of song thereof
WO2015127445A1 (en) * 2014-02-24 2015-08-27 Perkins Gilbert Newman Educational literary composition system
US9183820B1 (en) * 2014-09-02 2015-11-10 Native Instruments Gmbh Electronic music instrument and method for controlling an electronic music instrument
USD766325S1 (en) * 2014-09-02 2016-09-13 Samsung Electronics Co., Ltd. Display screen or portion thereof with graphical user interface
EP3243198A4 (en) * 2015-01-08 2019-01-09 Muzik LLC Interactive instruments and other striking objects
US20160240098A1 (en) * 2015-02-12 2016-08-18 Seoul National University R&Db Foundation Smart tablet-based neurofeedback device combined with cognitive training, method and computer-readable medium thereof
US9723253B2 (en) 2015-03-11 2017-08-01 Sony Interactive Entertainment Inc. Apparatus and method for automatically generating an optically machine readable code for a captured image
CN108140369A (en) * 2015-07-17 2018-06-08 乔凡尼技术公司 Music score, system and method
US9711121B1 (en) * 2015-12-28 2017-07-18 Berggram Development Oy Latency enhanced note recognition method in gaming
US9651921B1 (en) * 2016-03-04 2017-05-16 Google Inc. Metronome embedded in search results page and unaffected by lock screen transition
WO2018098406A1 (en) * 2016-11-28 2018-05-31 Intellivance, Llc Multi-sensory literacy acquisition method and system
US10170088B2 (en) 2017-02-17 2019-01-01 International Business Machines Corporation Computing device with touchscreen interface for note entry
US10776715B2 (en) 2017-04-28 2020-09-15 Microsoft Technology Licensing, Llc Artificial intelligent cognition threshold
US10002542B1 (en) * 2017-06-05 2018-06-19 Steven Jenkins Method of playing a musical keyboard
JP7193797B2 (en) * 2018-11-06 2022-12-21 任天堂株式会社 Game program, information processing system, information processing device, and game processing method
CN109994017A (en) * 2018-12-01 2019-07-09 李昱婷 A kind of bonusing method for supplementary music practice
US11341944B2 (en) 2019-02-19 2022-05-24 Nutune Music, Inc. Playback, recording, and analysis of music scales via software configuration
CN110781820B (en) * 2019-10-25 2022-08-05 网易(杭州)网络有限公司 Game character action generating method, game character action generating device, computer device and storage medium
US11056088B2 (en) * 2019-11-15 2021-07-06 Inmusic Brands, Inc. System and method for grouping audio events in an electronic percussion device
US11393359B2 (en) 2020-04-16 2022-07-19 Make Music Count Llc System to teach mathematics using virtual musical instruments and associated methods
CN111991809B (en) * 2020-07-17 2024-01-30 北京达佳互联信息技术有限公司 Audio generation method, device, server and storage medium
US11670188B2 (en) 2020-12-02 2023-06-06 Joytunes Ltd. Method and apparatus for an adaptive and interactive teaching of playing a musical instrument
US11893898B2 (en) 2020-12-02 2024-02-06 Joytunes Ltd. Method and apparatus for an adaptive and interactive teaching of playing a musical instrument
US11900825B2 (en) 2020-12-02 2024-02-13 Joytunes Ltd. Method and apparatus for an adaptive and interactive teaching of playing a musical instrument
CN112870689A (en) * 2021-03-18 2021-06-01 魔豆科技(中山)有限公司 Intelligent violin, interaction method thereof and computer readable storage medium

Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US634798A (en) * 1899-02-27 1899-10-10 John Bishop Drier.
US4331062A (en) * 1980-06-02 1982-05-25 Rogers Allen E Visual note display apparatus
US4366741A (en) * 1980-09-08 1983-01-04 Musitronic, Inc. Method and apparatus for displaying musical notations
US4386741A (en) * 1979-11-02 1983-06-07 Jagenberg Werke Ag Apparatus for the multiple unwinding of webs
US4416182A (en) * 1981-09-24 1983-11-22 Allen Organ Company Keyboard instrument teaching device
US4781099A (en) * 1981-11-10 1988-11-01 Nippon Gakki Seizo Kabushiki Kaisha Musical quiz apparatus
US4997374A (en) * 1989-05-19 1991-03-05 Simone John A Teaching device
US5107743A (en) * 1989-12-04 1992-04-28 Decker Tom W Piano teaching device and method
US5183398A (en) * 1990-06-01 1993-02-02 The Software Toolworks Apparatus and method for interactive instruction of a student
US5219291A (en) * 1987-10-28 1993-06-15 Video Technology Industries, Inc. Electronic educational video system apparatus
US5392682A (en) * 1993-03-24 1995-02-28 Mccartney-Hoy; Peter J. Computerized training device for learning to play a piano
US5540132A (en) * 1994-06-16 1996-07-30 Hale; Beverly M. Method and apparatus for teaching musical notation to young children
US5864868A (en) * 1996-02-13 1999-01-26 Contois; David C. Computer control system and user interface for media playing devices
US5886273A (en) * 1996-05-17 1999-03-23 Yamaha Corporation Performance instructing apparatus
US6066791A (en) * 1998-01-28 2000-05-23 Renarco, Inc. System for instructing the playing of a musical instrument
US6204441B1 (en) * 1998-04-09 2001-03-20 Yamaha Corporation Method and apparatus for effectively displaying musical information with visual display
US6252153B1 (en) * 1999-09-03 2001-06-26 Konami Corporation Song accompaniment system
US6284961B1 (en) * 2000-09-01 2001-09-04 Richard C. Kimmel, Jr. System for associating musical notation with keys on a musical instrument
US6337433B1 (en) * 1999-09-24 2002-01-08 Yamaha Corporation Electronic musical instrument having performance guidance function, performance guidance method, and storage medium storing a program therefor
US6347998B1 (en) * 1999-06-30 2002-02-19 Konami Co., Ltd. Game system and computer-readable recording medium
US6388181B2 (en) * 1999-12-06 2002-05-14 Michael K. Moe Computer graphic animation, live video interactive method for playing keyboard music
US6414229B1 (en) * 2000-12-14 2002-07-02 Samgo Innovations Inc. Portable electronic ear-training apparatus and method therefor
US6452822B1 (en) * 2001-04-26 2002-09-17 International Business Machines Corporation Segmented match line arrangement for content addressable memory
US6582309B2 (en) * 1998-07-14 2003-06-24 Konami Co., Ltd. Game system and computer-readable recording medium
US6607446B1 (en) * 1999-02-26 2003-08-19 Konami Co., Ltd. Music game system, game control method for the game system, and computer-readable memory medium

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3058051B2 (en) 1995-04-27 2000-07-04 ヤマハ株式会社 Musical amusement system
US5627335A (en) 1995-10-16 1997-05-06 Harmonix Music Systems, Inc. Real-time music creation system
JP3767106B2 (en) 1996-09-27 2006-04-19 ヤマハ株式会社 Keyboard instrument
US6072113A (en) * 1996-10-18 2000-06-06 Yamaha Corporation Musical performance teaching system and method, and machine readable medium containing program therefor
JP3204652B2 (en) 1998-10-30 2001-09-04 コナミ株式会社 Rhythm game device and operation device
US6225547B1 (en) 1998-10-30 2001-05-01 Konami Co., Ltd. Rhythm game apparatus, rhythm game method, computer-readable storage medium and instrumental device
JP3320381B2 (en) 1998-11-12 2002-09-03 株式会社プレイモア Amusement equipment
JP3017986B1 (en) 1998-11-26 2000-03-13 コナミ株式会社 Game system and computer-readable storage medium
JP2000237455A (en) * 1999-02-16 2000-09-05 Konami Co Ltd Music production game device, music production game method, and readable recording medium
JP3088409B2 (en) 1999-02-16 2000-09-18 コナミ株式会社 Music game system, effect instruction interlocking control method in the system, and readable recording medium recording effect instruction interlocking control program in the system
JP3361084B2 (en) 1999-09-08 2003-01-07 コナミ株式会社 GAME SYSTEM AND COMPUTER-READABLE STORAGE MEDIUM FOR IMPLEMENTING THE SAME
JP3320700B2 (en) 1999-09-08 2002-09-03 コナミ株式会社 Game system and computer-readable storage medium for realizing the game system
JP3718611B2 (en) 1999-11-11 2005-11-24 コナミ株式会社 Dance game equipment
JP3743231B2 (en) * 1999-11-26 2006-02-08 ヤマハ株式会社 Song data display control apparatus and method
JP2001162031A (en) 1999-12-10 2001-06-19 Namco Ltd Music game device
JP3329786B2 (en) 2000-02-29 2002-09-30 コナミ株式会社 Dance game equipment
JP2001246167A (en) 2000-03-06 2001-09-11 Namco Ltd Game device and computer readable recording medium storing program for game
US20010036620A1 (en) * 2000-03-08 2001-11-01 Lyrrus Inc. D/B/A Gvox On-line Notation system
JP3729019B2 (en) 2000-03-29 2005-12-21 松下電器産業株式会社 Musical instrument education system
JP2001276422A (en) 2000-03-31 2001-10-09 Namco Ltd Music playing game machine
US6870085B2 (en) * 2000-08-03 2005-03-22 Maccutcheon Jane S. Music teaching system and method
US6740802B1 (en) * 2000-09-06 2004-05-25 Bernard H. Browne, Jr. Instant musician, recording artist and composer
JP3719124B2 (en) * 2000-10-06 2005-11-24 ヤマハ株式会社 Performance instruction apparatus and method, and storage medium
US6791568B2 (en) * 2001-02-13 2004-09-14 Steinberg-Grimm Llc Electronic color display instrument and method
US6660922B1 (en) * 2001-02-15 2003-12-09 Steve Roeder System and method for creating, revising and providing a music lesson over a communications network
US20020144587A1 (en) * 2001-04-09 2002-10-10 Naples Bradley J. Virtual music system
US7030307B2 (en) * 2001-06-12 2006-04-18 Douglas Wedel Music teaching device and method
US7928310B2 (en) * 2002-11-12 2011-04-19 MediaLab Solutions Inc. Systems and methods for portable audio synthesis
US20060058101A1 (en) * 2004-09-16 2006-03-16 Harmonix Music Systems, Inc. Creating and selling a music-based video game
US8079907B2 (en) * 2006-11-15 2011-12-20 Harmonix Music Systems, Inc. Method and apparatus for facilitating group musical interaction over a network
US20090075711A1 (en) * 2007-06-14 2009-03-19 Eric Brosius Systems and methods for providing a vocal experience for a player of a rhythm action game

Patent Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US634798A (en) * 1899-02-27 1899-10-10 John Bishop Drier.
US4386741A (en) * 1979-11-02 1983-06-07 Jagenberg Werke Ag Apparatus for the multiple unwinding of webs
US4331062A (en) * 1980-06-02 1982-05-25 Rogers Allen E Visual note display apparatus
US4366741A (en) * 1980-09-08 1983-01-04 Musitronic, Inc. Method and apparatus for displaying musical notations
US4416182A (en) * 1981-09-24 1983-11-22 Allen Organ Company Keyboard instrument teaching device
US4781099A (en) * 1981-11-10 1988-11-01 Nippon Gakki Seizo Kabushiki Kaisha Musical quiz apparatus
US5219291A (en) * 1987-10-28 1993-06-15 Video Technology Industries, Inc. Electronic educational video system apparatus
US4997374A (en) * 1989-05-19 1991-03-05 Simone John A Teaching device
US5107743A (en) * 1989-12-04 1992-04-28 Decker Tom W Piano teaching device and method
US5183398A (en) * 1990-06-01 1993-02-02 The Software Toolworks Apparatus and method for interactive instruction of a student
US5392682A (en) * 1993-03-24 1995-02-28 Mccartney-Hoy; Peter J. Computerized training device for learning to play a piano
US5540132A (en) * 1994-06-16 1996-07-30 Hale; Beverly M. Method and apparatus for teaching musical notation to young children
US5864868A (en) * 1996-02-13 1999-01-26 Contois; David C. Computer control system and user interface for media playing devices
US5886273A (en) * 1996-05-17 1999-03-23 Yamaha Corporation Performance instructing apparatus
US6066791A (en) * 1998-01-28 2000-05-23 Renarco, Inc. System for instructing the playing of a musical instrument
US6204441B1 (en) * 1998-04-09 2001-03-20 Yamaha Corporation Method and apparatus for effectively displaying musical information with visual display
US6582309B2 (en) * 1998-07-14 2003-06-24 Konami Co., Ltd. Game system and computer-readable recording medium
US6607446B1 (en) * 1999-02-26 2003-08-19 Konami Co., Ltd. Music game system, game control method for the game system, and computer-readable memory medium
US6347998B1 (en) * 1999-06-30 2002-02-19 Konami Co., Ltd. Game system and computer-readable recording medium
US6252153B1 (en) * 1999-09-03 2001-06-26 Konami Corporation Song accompaniment system
US6337433B1 (en) * 1999-09-24 2002-01-08 Yamaha Corporation Electronic musical instrument having performance guidance function, performance guidance method, and storage medium storing a program therefor
US6388181B2 (en) * 1999-12-06 2002-05-14 Michael K. Moe Computer graphic animation, live video interactive method for playing keyboard music
US6284961B1 (en) * 2000-09-01 2001-09-04 Richard C. Kimmel, Jr. System for associating musical notation with keys on a musical instrument
US6414229B1 (en) * 2000-12-14 2002-07-02 Samgo Innovations Inc. Portable electronic ear-training apparatus and method therefor
US6452822B1 (en) * 2001-04-26 2002-09-17 International Business Machines Corporation Segmented match line arrangement for content addressable memory

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8642871B2 (en) 2008-11-24 2014-02-04 Piano Matchmaker Llc Instructional music reading and instrument playing system and method
US20120192700A1 (en) * 2009-06-01 2012-08-02 Starplayit Pty Ltd Music display system
US20110239845A1 (en) * 2010-03-31 2011-10-06 Yamaha Corporation Musical score display apparatus and program for realizing musical score display method
US8497416B2 (en) * 2010-03-31 2013-07-30 Yamaha Corporation Musical score display apparatus and program for realizing musical score display method
US20120090446A1 (en) * 2010-04-20 2012-04-19 Sylvain Jean-Pierre Daniel Moreno System and method for providing music based cognitive skills development
US8536436B2 (en) * 2010-04-20 2013-09-17 Sylvain Jean-Pierre Daniel Moreno System and method for providing music based cognitive skills development
US11386803B1 (en) 2010-04-20 2022-07-12 Sylvain Jean-Pierre Daniel Moreno Cognitive training system and method
US20190228754A1 (en) * 2018-01-23 2019-07-25 Roland VS LLC Generation and transmission of musical performance data
US10482858B2 (en) * 2018-01-23 2019-11-19 Roland VS LLC Generation and transmission of musical performance data

Also Published As

Publication number Publication date
US20030151628A1 (en) 2003-08-14
US7799984B2 (en) 2010-09-21
US20060107819A1 (en) 2006-05-25
US7174510B2 (en) 2007-02-06
WO2004038690A1 (en) 2004-05-06
US20060252503A1 (en) 2006-11-09
US7739595B2 (en) 2010-06-15
AU2003301598A1 (en) 2004-05-13

Similar Documents

Publication Publication Date Title
US7739595B2 (en) Interactive game providing instruction in musical notation and in learning an instrument
KR100856928B1 (en) An interactive game providing instruction in musical notation and in learning an instrument
US20040137984A1 (en) Interactive gamepad device and game providing means of learning musical pieces and songs
JP4075565B2 (en) Music score display control apparatus and music score display control program
US5746605A (en) Method and system for music training
JP3149574B2 (en) Karaoke equipment
US6423893B1 (en) Method and system for electronically creating and publishing music instrument instructional material using a computer network
US7754955B2 (en) Virtual reality composer platform system
JP4111004B2 (en) Performance practice device and performance practice program
CN109345905B (en) Interactive digital music teaching system
US20110191674A1 (en) Virtual musical interface in a haptic virtual environment
WO2001061673A1 (en) System and method for variable music notation
Menzies et al. A digital bagpipe chanter system to assist in one-to-one piping tuition
Fober et al. Imutus-an interactive music tuition system
McCord Music software for special needs: Music educators faced with adapting their instruction for children with special needs can find effective tools in computer software
Crow Music-related ICT in education
AU2002337891A1 (en) An interactive game providing instruction in musical notation and in learning an instrument
Bowyer A new approach to computer-assisted instruction in music theory for elementary and middle school children
Jacoby iShed: Interactive software for the training of novice jazz improvisers
Askenfeld D. Fober, S. Letz, Y. Orlarey Grame-Centre national de création musicale {fober, letz, orlarey}@ grame. fr A. Askenfeld, K. Falkenberg Hansen, E. Schoonderwaldt KTH-Royal Institute of Technology
JP2020003721A (en) Musical instrument performance practice device and program for musical instrument performance practice
GB2254954A (en) Teaching device for musical intsruments
Schachter The use of new technology for piano instruction in a structured learning environment
Reyes GLozart: A 3D Graphics Piano Aid
Chau Computer and Music Pedagogy

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION