CA1331653C

CA1331653C - Interactive video based communication system

Info

Publication number: CA1331653C
Application number: CA000605961A
Authority: CA
Inventors: Michael Paolini
Original assignee: Individual
Current assignee: Individual
Priority date: 1988-07-19
Filing date: 1989-07-18
Publication date: 1994-08-23
Anticipated expiration: 2011-08-23
Also published as: US5276866A

Abstract

ABSTRACT
An interactive video based communication system provides apparatus and methods for user guided information retrieval in video frame format and transmission of the selected and retrieved information to a remote location for subsequent review and analysis. The storage of audio information in video frame format permits real time information compression allowing a one minute presentation to be transmitted in a fraction of a second.

Description

INTERACTIVE vIDEO BASED COMMUNICATION sYSTEM

This invention relates to apparatus and methodology for providing an interactive video based communication system, and more particularly, a system for interactively selecting video based data comprising image, audio and/or text information from a data storage facility and transmitting this information as a video based signal to a remote location for subsequent viewing and analysis.
It is an object of the present invention to permit a user to interactively select portions of video based data from a database storage facility. The video based data will contain picture image information, audio information, and/or text and graphic information. The selected portions are transmitted pursuant to the present invention to the user for viewing and analysis.
It is an additional object of the pre~ent invention to combine image, text and audlo information into a video signal for retrieval by the user.
It is another object of the present invention to permit the ` transmission of~high definition video images.
`~`' It i- still onother object of the present inventlon to tagthe user selected video based data in a manner to control and monitdr its transmission and reception.
Another ob~ect of the present invention is the provision of an interaotive~video bas-d communication system directed to the storage, retrieval;~nd display of video based data pertaining to the sale of rea~l property.
An additional object of the present invention is the provision of an interactive video based communication system directed to the storage, retrieval and display of video based data pertaining to purchasable goods and services.
:~

-2- ~

-1~

133~ ~3 60538-l0l0 Still another object of the present invention ls the -~
provision of an lnteractive video based communlcation sy4tem directed to the controlled storage, retrieval and dlsplay of video based data pertaining to resorts or vacation locations as used by a network of travel agents.
The above and other ob~ects of the present lnvention are realized ln a specifle 1llustratlve program controlled interactive video based proce6sor that operates in concert wlth lnput, transmi#slon and dlsplay apparatus to access user -~
selected topics ln the form of video based data; confirm the ;~
quallty of the selected data and transmit the selected data in the form of a video 6ignal to a remote location; recelve and store the transm1tted data at the remote location for subsequent review and analy~
In aecordance with varying aspeets of the present lnventlon, the selected data i~ interaatively quallfied by the ^~
user p.ior to transmission, thu~ permitting the user to augment or eanc-1 hls request lt the se1eeted data 1B un~atisfactory. ;~
In additlon, access to the system is by coded entry, creating a ~20 ~ s1mp11f~1-d means tor tracking use and insuring proper transmission and billing for the system service.
~- Aceording to one aspect, the inventio~ may be ! ;~
summarlzed as an apparatus for lnteractlve video based eommun1cation eomprls1nq~ a t-lephone based, multl-user access means for descrl~lng a request for lnfor~ation; a random access ~ -lntor atlon storag- m-ans for storlng plural infornation tra~cks~ data processing eans for recelvlng said request for information and selectively searching based on sald request and ,~ , , generatlng identlfying lnformation indicatlng infor~ation track~ ln sald intormatlon ætorage means responsive to said requ-st for informatlon; a telephone based communlcation link means for configuring a plurality of remotely located receivers `, ,5-' ~ 3 133~ 6~3 `-;

for selective ~torage or di~play of ldentifled informatlon tracks; and transmittlng means for transmittlng ~ald identifled information tracks in video frame format to a plurallty of ~ald remotely located recelvers, whereln sald remotely located recelvers include frame grabber means for selectively acquirlng transmitted lnformation tracks re~ponslve to ~ald search request~
According to another aspect, the invention may be ~ummarized as ln a system for virtual real time vldeo frame ~-`
formatted com~unlcatlon wherein a communlcated nessage lncludes lmage lnformatlon encoded and stored on at least one vldeo frame, and audio lnformatlon encoded and ~tored on at least one ~ -~ - . , video frame whereln sald audio lnformatlon 18 coapressed ln -~ -real tlme, said sy~tem coDprlslng a random acces~ data storage mean~ for storlng lndlvldual image and/or audio lnformatlon fornlng communlcated messages ln vldeo frame for~at, a data ~ -proce~slng oeans for selectlvely identlfylng and retrleving sald com~unlcated ~es~age from sald randoD acce~s data storage; 1 plural~receptlon~-ans~ a~flrst transDisslon means for ~ `
~20 interactive co~munlcatlon~wlth said data processlng ~ean~ and ~ - -for ¢onflgurlng sald plural reception means to selectlvely recelve the comounlca,ted mejssages~ and a second, transDlssl!on eans aonnected to ~ald data processing means for transmlttlng retrieved communlcat-d ~-ssages to said plural r-ceptlon ~eans in video frame format, for re-expansion and dlsplay at said p~ura} receptlon means ~ -~
The foregoing features of the present lnventlon may be more fully understood from the following dlscusslon of a - ~ :',: `
speclflc, lllustrative embodlment thereof, presented - ~--herelnbelow ln con~unctlon wlth the accompanylng drawlng, ln whlch, 3a i: !
,, -~
?~

1~3~ 6~3 Figure 1 i5 a block diagram of video ba6ed data stora~e, retrieval and transmls~ion sy~te~s in accordance with the present invention;
Flgure 2a iB a block diagram of the program controlled receiver and dl~play ~y~tem used in con~unction with ~ -the apparatu~ of Fig. 1 and in accordance with the ~ub~ect invention;

:"' '~ `

133~ ~3 Fig. 2b is a block diagram of an alternative arrangement for the receiver system.
Figure 3 is a logic flow chart for the eontrol functions of the transmission system in accordance with the present invention; and Figure 4 is a logic flow chart for the control functions of the recovering system in accordance with the subject invention.
First briefly discussing the invention in overview, the interactive communication system herein described providec user access to a database of in~ormation and the capability to select specific portions of the database information for his/her subsequent reception. The database will contain video, audio, and text information organized in a manner to allow the user to interactively ~cull out~ the information tracks he/she desires. The selected portions of data are tagged (I.D.) pulled out of memory and transmitted to a user location. The selected data, aIthough containing both image and audio information, is transmitted as a video signal at the prevailing industry-accepted standards (e.g. 30 frames per second for video images).
The audio portion of the selected data is compressed from real time format for video-signal transmisaion (e.g. 10 seconds of audio information is stored in one ~video~ frame and transmitted at 30 rames/sècohd thùs providing a 300'1 time compression ratio of the audio signal). In this regard, a video frame is a bit-map of pixel intensity values, and is set by convention at 512 x 480 pixel locations (a pixel is a picture image element).
The audio data is formatted and stored in pixel form on the video frame.
At the user loca~ion, the selected data is received, processed, stored and directed to specified output devices, such as monitora, speakers and/or printers. Since the received data 133~ ~3 is stored, the user can selectively access the data during non-reception intervals. This allows the transmission system to simultaneously support multiple users.
With the above overview in mind and referring to Fig. 1, the transmission segment of the present invention is shown in block diagram form. More specifically, a component based system is presented with modem 110 connected to local phone service.
The ~modem~ device (modem is an abbreviation for modulator/de-modulator) contains E~E se well known circuitry for converting data requests received via the telephone wire into digital information suitable for processing vis-a-vis a digital computer. The modem also translate~ digital information from the computer into a signal compatible with the teiephone service. An interactive voice synthesizer/receiver (bloek 120) is linked with the modem and provides voice output of digital information from the central processor, 100.
The central processor unit (~CPU~) i9 designated by numeral 100. The CPU is a digital data processor and operates on Der well known principles of digital computing. System instructions are coded in a form of progra~ming language recognizable by the specific hardware employed. More particularly, the CPU has stored program steps that dictate its operation in response to specific inputs. Any type of programming language can be used as long as it is compatible with the CPU, and input/output devices. The CPU is a microproce~sor controlled computer such as, inter alia, mainframe, mini, or personal computer, as specifically programmed for the functions delineated below.
CPU, 100 is in communication with the various system components via address and data busses, 130 and 140 respect-ively. These communication busses permit the transmission of instructions and data by per se well known means between the - ~33~ ~3 memory and output devices of the system via industry standard plug compatible components.
In the present system, the mass storage of information is held in memory database 150. This database ear, be formed by conventional data storage devices, such as semi-conductor, magnetic disk, or preferably by an optical storage system. The advantage to optical storage of information resides in its non-volatility and its relatively inexpensive capital and operating costs. More particularly, an optical data storage and retrieval device known as a WORM (i.e., write once, read many) provides a data storage medium consisting of micron size pits on the surface of a disk or sheet of polymeric material. These pits are organized into ~information tracks~ forming digitally encoded information (i.e. in binary code~). Each sheet or disk contains billions of bits of information and provides a file to a database library. In addition, each disk includes a track that defines and summarizes the data stored therein, thus providing a simplified method for evaluating the quality of data without a full analysis of the stored information. Data is retrieved off the disk or sheet by uae of a scanning laser. The laser is sensitive to the change in depth of the pits in the polymeric surface and thereby generates a signal corresponding to these pits while rapidly tracking the disk's surface.
'The data's'tored ~in this format can ~e of audio, text and/or image information. For example, an information track represents a series of ~frames~ in that each frame contains binary data sufficient to represent an image at 512 x 480 pixel locations for a total of approximately 25,000 pixels. The optical scanner generates a video based signal comprising 30 frames per second of video frame information; many of these video frames are pre-formatted to audio or text data and suitably tagged, thus generating a heterogeneous signal blending audio and text information with picture images, but all in video frame format.
Although discussed above as a discrete device, the database memory can be fully integrated into, and part of, the CPU.
Furthermore, stored data can be in analog format, as well as in blended analog/digital or pure digital format.
A data search as requested by the user is controlled by the CPU (See Fig. 3 and the discussion hereinbelow). A se~rch summary is culled from the Database 150 and stored in volatile memory 155. Memory 155 can be one of many well known memory devices, including arrayed random access memory semi-conductors (RAM). The information stored in memory 155 is transient in that its purpose is to guide the user in completing his search.
The user reviews these qualitative results and decides whether to quit his search, extend his search, or receive the search in its present form. Once this decision is made the transient data in memory 155 is discarded.
The selected data resulting from the user defined search is culled from the database in the form of a video signal, even though portions or all of the information is audio or text.
This video signal is modulated, 160, and transmitted at a carrier frequency by any EÇ~ se well known transmission means, 170.
Referring now to Fig. 3, CPU 100 contains stored program steps per the i`ndicated flow chart logic. A system run is initiated by user entry into the system via telephone (modem) or similar. The user enters his identification code (I.D.) and the specific search request being made. If the I.D. is positive, the CPU enters a ~tag~ value which operates to identify the search request for all subsequent processing. The requested search is performed through the database coupled to the CPU (In Fig. 1, block 150). A database search is accomplished by ~ÇE se well known means in that responsive .,;

1~3~ 6~i3 tracks of information are ~read~ and identified, 460. In addition, a search summary is prepared, 470, which contains the number of responsive tracks found in the data search, and a brief synopsis of the information within these data tracks.
This summary is stored in a separate memory cache.
Test 480 determines whether the search has been completed in terms of the user requests. If affirmative, the logic proceeds to block 490, and the search ~summary~ is conveyed to the user from memory. This is accomplished by telephone line, via the interactive voice synthesizer (Fig. 1, Block 120). Test 500 determines the need for additional searches; if no, the user is queried on whether the responsive tracks of data representing the requested search are to be transmitted to a user-specified location, 510. If ye~ to block 510, then the system transmits the tagged search results to the remote receiver specified by the user, block 515. The account is charged, 520, and the run terminated, 530.
In operation, the transmitting system is continually receiving data search requests by subscription based users.
These interactive requests are handled in mixed sequential fashion, as the system supports multiple requests concurrently.
The heterogeneous nature of the stored data permits random access of the selected video frames from the database. Since the data is trànsmitted in full video format, information compression from real time permits the system to simultaneously serve multiple users.
Now referring to Fig. 2a, the reception system of the present invention is depicted. The transmitted signal generated by the transmitter, 170 (Fig. 1) is collected by receiver 210.
The mode of transmission is not particularly limiting; the transmission is video based on industry or government set standards, e.g. 30 video frames per second. Also the 133~ 6~3 .
transmitter and receiver must be compatible to the particular transmission system selected. Possible modes of transmission include microwave broadcast, cable, UHF/VHF, and satellite systems, or combinations thereof. The incoming signal is demodulated (block 220) by ~ÇL se well known means, and combined in a multiplexer, 240, with information supplied via telephone service. The telephone service input permits, through modem 230, the user and/or service operator to verify that the receiver is properly configured to receive the transmitted data, that the receiver is authorized to receive the transmitted data, and to confirm reception. In this way, multiplexer 240 acts as a gate to the received data prior to conveying this data into memory under the receiver's control.
If the tagged data, as transmitted, properly matches the code of the receiver system, the transmitted data is converted to a digital signal by analog to digital converter, 245 and entered into Memory 205 via address and data busses 250 and 260 respectively. In a manner more fully presented below, CPU 200 directs the received data to various output devices, such as video monitor, audio speaker, print~r and/or video tape recorder (VTR) .
Throughout the process, the data is in the form of individual vid~o frames wherein each frame is labeled or tagged.
This tag incorporates information on data type (i.e. image, audio, or text). TAis permits the CPU to direct the tagged frame to the proper output device in the proper sequence for the requested presentation. If the data presentation is complex, the instructions thereto (i.e., ou~put algorithms) can be stored on a dedicated video frame and subsequently used by the CPU to control the output. For example, the incoming signal may have an initial frame containing output control instructions, a second frame containing a picture image, followed by a series of 1~3~ 6~3 frames containing audio information, somewhat analogous to a slide presentation. CPU 200 recognizes the initial output control frame, reads the control algorithm and directs the second, labeled image frame with the picture-image to image frame grabber 270 with instructions to ngrab~ and circulate that image on monitor 300 for a certain length of time, this time corresponding to the duration of the audio output. The subsequent frames containing the audio information are directed to audio frame grabber, 271, with instructions to drive speaker 275 with the audio signal. This directed output via CPU 200 is by E~E se well known means of driving peripheral devices.
As mentioned, image frame grabber 270 ~grabs~ the picture image frame and stores the frame in RAM memory. This image is directed to monitor 300 by controller 278 wherein it ~refreshes~ the screen display at a rate of 30 frames per second providing the picture image without associated image ~flicker~.
This ~ingle imagQ frame is held on the display for a time period determined for that particular presentation.
The audio frame grabber, 271 ~grabs~ the video frames containing audio information and produces an audio output therefrom. In this regard, the grabbed frame is ~read~
providing a certain time length of audio information to the spe~ker, e.g. 10 seconds per frame, in conjunction with the concurrently running vidéo image on monitor 300. Although, the audio and image frame grabbers are diagrammatically discrete, this is for illustration of the above principles. A single frame grabber can be employed to perform the above functions.
An alternative reception process is diagrammed in Fig. 2b.
More specifically, the reception system depicted in Figs. 2a and 2b are the same through to multiplexer, 240. In Fig. 2b, the output of multiplexer 240 is in the form of demodulated broadcast video, tagged for that receiver location and at 1331 6~3 ., prevailing industry set standards, e.g. 30 frames per second.
This signal is fed to memory device 340, in unconverted analog form. An example of memory device 340 is an analog ~frame grabber~ floppy disk drive which is capable ~f storing video frames on individual tracks in a magnetic recording medium. The output of the recorded tracks is directed by controller 360, to the various output devices 350, e.g. speaker and monitor, pursuant to instructions found on each frame. Again controller 360 and memory device 340 are only shown as discrete devices for understanding of the principles involved.
The systems in Fig. 2a and 2b both incorporate input 280, allowing a user to direct and control the output. This input can be in the form of keyboard or similar wherein the u er can enter commands to the system.
Referring now to Figure 4, a logic flow chart is presented for the video receiver system. Althaugh in practice, the receiver would be continuously receiving and confirming coded data, this is conceptually presented as initiated by the reception of a transmission compatible with the system, 600, and confirmation of the tagged data as properly matched and expected by th~ system, Test 610. }f the stream of data, in the form of video ~rames, does not match expected system codes, it is rejected (No to t-st 610) and th- run is conceptually complete, 720. Alternatively, if the incoming video frames are properly tagged, or a portion thereof, then the system recognizes the frames and stores the data in the form of video frames in memory 620. As discussed above, this memory can be one of many industry recognized systems including magnetic based floppy disk and hard disk devices.
As stored, the video based images are now available for review by the system user, 630. Test 640 queries whether the user desires a real time display o~ the stored data. If yes, 133~ ~3 the system directs the video frames representing picture images to the monitor 650 in a manner corresponding to that particular presentation. The video based audio information is reformatted to audio output and similarly directed to output via loudspeaker, either discrete or integrated with the monitor.
Test 660 determines whether the user desires to store the data on a hard storage device, such as a VTR. If ~yes~, then the output is directed to hard storage, 670. Simi].arly, Test 680 determines whether the operator desires to print individual video frames. If ~yes~ the desired frames are identified, 690, and the output directed to a suitable printer device, 700.
A second run through the same data is possible, Test 710, with the run conceptually completed at 720. Although the logic shown is hierarchial in form, this is chosen mostly for enhancing explanation of the logic path. It may be more desirable to present the various options in the system to the user in concurrent ~menu driven~ format by ~ se well known techniques.
The properties of the above system will be more fully recognized as operating in the context of the following example.

.
EXAMPLE
In the Real Estate market, Realtors operate locally through agents. These~`agents take potential customers (homebuyers) as prospects to houses listed or advertised for sale on the market.
In addition, these agents continually visit other houses on the market and consult with listing æystems, such as the Multiple Listing Service (MLS) to measure the inventory of houses on the market and to match suitable houses to prospects. The present invention operates to increase the efficiency of the agent in performance of these functons.

More particularly, a client visits an agent at his office, 133~ ~3 requesting information about four bedroom homes in a given neighborhood and within a certain price range. The agent contacts an operating service equipped with the present invention via conventional telephone lines, ~rovides the access code to the system, and describes his search request. This telephone link directly accesse~ the system. For example, the system iteratively asks the agent to characterize his search reguest by a series of choices, such as commercial or residential, condominium or detached home, etc. to which the agent responds by touch tone entry of designated numbers.
once the search is properly characterized and the agent's access code recognized, the system identifies the informat$on -tracks responsive to the requested search. These tracks comprise image, audio, and text data in ths form of video frames. In addition, the requested data is tagged with the agent's access code, location code, and other information to facilitate system operation. Finally, a data search ~ummary is separately stored, summarizing the contents of the responsive information tracks.
The system pr-sents the summary of the data retrieved by system to the agent over the telephone. This summary may include the number of houses found in the search, street address, etc. If the agent is satisfied with the search and " ~ , wishes to view the information retrieved, the system is instructed to transmit the responsive tracks to the agent's location.
These tracks are transmitted by microwave broadcast in the form of video frames at industry set standards, e.g. 30 frames per second. In this way, the images of 30 different houses can he transmitted to the agent's location in 1 second.
Alternatively, information on houses for sale can be prepackaged including separate image frames for varying views of the house, ..

1331~3 intermixed with audio and/or text information on the house. A
one-minute formal presentation for a house comprising three different views (3 image video frames) with 60 seconds of audio (in the form of 6 video frames, each containing 10 seconds of audio) can be transmitted ~o the agent's location in less than 1/3 of second (total 9 frames of video signal at 30 frames per second transmission rate3.
The agent's location is equipped and configured to receive the microwave broadcast signal comprising designated frames of video based data corresponding to presentations on four bedroom homes at the requested price level. The transmitted data is received, identified, and stored in receiving system memory.
The agent and his client can review each house, or house presentation at their leisure, store the video information on a video tape recorder for later review, or print individual houses (image video frames) for the client to consider on his own. For the real estate agent, the system provides tremendous access to detailed information about the housing market with minimal time.
In addition, since the system compresses real time presentations to a small fraction of the time for transmittal, multiple users can simultanQously access~the system.
In a similar and simultaneous manner, the present invention is applied to support a network of travel agents. In this application, tha travel~agent inquires about specific locations for vacation travel. Video based presentations on hotels, beaches, etc. are transmitted to the agent for review.
Another variation of the present invention is its application for goods-services purchasing, as exemplified by home-buying services. For this application, the customer will interactively requests data on a certain class of products. The description/presentation of these products is transmitted in ~33~ ~3 video frame format to the specified buyer for his subsequent review.
An additional application of the above invention involves high definition television (HDTV). In this case, the system operates to expand the transmitted video signal in real time, so that, upon compression at the user's site, the image has much higher resolution. More particularly, an high resolution image frame containing 100,000 pixels (picture image elements) can be sequenced into four segments of pixal information (e.g., 2S,000 pixels) and transmitted as four separate video frames in industry standard 512 x 480, 30 frames per second format. At the point of reception, these four frames are recombined into the original, high resolution image.
It can be seen that the above system provides significant flexibility in tailoring the transmitted signal to the particular needs of the user. For example, multiple languages can be intergrated into a presentation with a user ~pecifying his languag- of choice. Additionally, higher guality sound, and multichannel sourcing (stereo) is asily established in video frame format for subseguent retrieval.
Accordingly, the above described invention has been shown to p-r it th- interactive selection of information, the retri-val of this information in video based format for transmission to alremote, identifiable location for storage and subseguent review. The above described arrangement is merely illustrative of the principles of the present invention.
Nu~erous adaptations and modifications thereof will be readily apparent to those skilled in this art without departing from the spirit and scope of the present invention.

Claims

THE EMBODIMENTS OF THE INVENTION IN WHICH AN EXCLUSIVE
PROPERTY OR PRIVILEGE IS CLAIMED ARE DEFINED AS FOLLOWS:

1. An apparatus for interactive video based communication comprising- a telephone based, multi-user access means for describing a request for information; a random access information storage means for storing plural information tracks data processing means for receiving said request for information and selectively searching based on said request and generating identifying information indicating information tracks in said information storage means responsive to said request for information; a telephone based communication link means for configuring a plurality of remotely located receivers for selective storage or display of identified information tracks; and transmitting means for transmitting said identified information tracks in video frame format to a plurality of said remotely located receivers, wherein said remotely located receivers include frame grabber means for selectively acquiring transmitted information tracks responsive to said search requests.

2. An apparatus as in claim 1 t wherein said data processing means includes tagging means for identifying a type of information found on said information tracks.

3. An apparatus as in claim 2, wherein said tagging means further comprises a means for tagging said identified information tracks in a manner permitting selective reception by said remotely located receivers of the tagged information tracks.

4. An apparatus as in claim 1, wherein each of said remotely located receivers comprises a gate means for selective storage of said identified information tracks transmitted by said transmitting means.

5. An apparatus as in claim 1, wherein said information tracks comprise high resolution image information wherein plural individual video frames each form a fractional portion of said high resolution image.

6. An apparatus as in claim 1, wherein said information tracks comprise compressed audio information.

7. In a system for virtual real time video frame formatted communication wherein a communicated message includes image information encoded and stored on at least one video frame, and audio information encoded and stored on at least one video frame wherein said audio information is compressed in real time, said system comprising: a random access data storage means for storing individual image and/or audio information forming communicated messages in video frame format, a data processing means for selectively identifying and retrieving said communicated message from said random access data storage;
plural reception means; a first transmission means for interactive communication with said data processing means and for configuring said plural reception means to selectively receive the communicated messages; and a second transmission means connected to said data processing means for transmitting retrieved communicated messages to said plural reception means in video frame format, for re-expansion and display at said plural reception means.

8. The system of claim 7, wherein the first transmission means is a telephone link.

9. The system of claim 8, wherein said communicated message includes plural frames comprising a high definition image.

10. The system of claim 9, wherein said plural reception means comprise audio expansion means for converting said compressed audio signal in real time.

11. The system of claim 10, wherein said reception mean further comprise a frame grabber means for storing said transmitted communicated message.