WO1990016131A1

WO1990016131A1 - Digital video recording

Info

Publication number: WO1990016131A1
Application number: PCT/GB1990/000925
Authority: WO
Inventors: Michael Joseph Kemp; Charles Duncan Maclean
Original assignee: Spaceward Limited
Priority date: 1989-06-15
Filing date: 1990-06-15
Publication date: 1990-12-27
Also published as: GB8913803D0

Abstract

Apparatus for storing a video signal includes means for repeatedly determining a portion of the picture area which contains moving information. Only the portion of the video signal which corresponds to this area containing moving information is then stored. The video signal comprises luminance and chrominance components and may have an associated matte signal. The apparatus includes means for recording the luminance and chrominance components and the matte in synchronism on a common record medium. A camera produces a video signal at an output and means for recording this signal is also provided. The camera also includes means for detecting the location and/or orientation of the camera and generating signals dependent thereon. These signals are recorded in synchronism with the video signal. Apparatus for generating a video signal is used to provide a composite picture in combination with a video signal which has camera location and/or orientation data associated with it. The system may include means for computing objects to be included in the video signal and will also include means for producing a matte associated with that computed object for each pixel which that object intersects.

Description

DIGITAL VIDEO RECORDING

This invention relates to digital video recording.

A digital video signal consists of a stream of data

representing a video waveform. This may be recorded on tape (digital video tape recorder) or in solid state storage devices, e.g. RAM. Several such machines exist on the market and may be referred to as solid-state video recorders.

Due to the amount of data, it requires very large amounts of memory to record any significant duration of video. A video frame to the CCIR 601 television standard comprises typically 720 by 576 samples for luma and 360 by 576 samples for each of the two colour difference signals. This represents 829,440 samples per frame, and each sample is typically of 8 bits accuracy. Thus 25 frames (representing one second of video) occupies around 20 MegaBytes of memory.

If it is desired to record a parallel matte channel (to provide a 'travelling matte' to accompany the video) this can represent a further 10 MegaBytes of storage per second of video.

Methods of reducing the data rate exist and one approach is to 'compress' the data by various techniques of analysing patterns and repeated information in the signal during the recording process, saving the compressed data, and reconstituting the data in real time during playback. The recording can often not be in real time due to the difficulty in analysing the incoming signal.

Another approach is to recognise that moving information is contained in only one area of the picture. Therefore if the background is saved as a still picture, it is necessary to record only the area encompassing the moving region. Thus only

relatively small areas may need to be recorded. They may be played back by being 'matted' over the still frame or more simply just by cutting (switching) to the still frame at the edge of the moving region. Where matting is used a travelling matte has to be recorded in parallel with the moving overlay, but no additional background needs to be recorded.

The known techniques utilise a fixed rectangular area to define the picture portion containing the moving image. In accordance with one aspect of this invention, the picture portion containing the moving image is repeatedly re-specified, preferably on a frame by frame basis. Thus a foreground object which is intended to appear as if from the distance and move towards the camera will only occupy a small amount of memory while distant as its bounding box is small, but as it gets closer it will require more and more memory per frame until perhaps eventually requiring the maximum amount of memory described above if its bounding box becomes equal to the full screen size.

If the full moving image is generated by computer, it is often done on a frame by frame basis. The apparently moving object can be generated along with a matte for each frame, and can be either stored along with its matte, or presetted over a static background.

In the former case, the matte is preferably stored as well as the colour image over the area of a bounding rectangle which encompasses the object and which is determined by the computer for each frame. During replay the colour area outside the rectangle (which is not recorded) is replaced with any arbitrary data (usually representing black for simplicity) and the matte area outside the rectangle is replaced with a value representing transparency.

Thus the colour data outside the rectangle is never seen once matted over a scene in post-production.

In the latter case, a single still is recorded representing the original background. This is used to replace the data outside the box for each frame, providing a full screen image iiώistinguiεhable from the moving image which would have been seen had each frame been recorded whole.

If the original image to be recorded is real 'live' video rather than computer generated, for this technique to be used it is necessary to derive a bounding box for each frame as it arrives at the recording machine. m the (fairly cannon) case that the live video is shot against a blue background machines exist which will derive a real-time travelling matte from the camera signal, and this is often recorded in parallel with the colour image for later use in post-production, e.g. on a second VTR running in synchronism with the main colour VTR.

We also propose receiving a colour signal and a matte signal. and deriving a bounding box from the matte information on a frame by frame basis. Then only the rectangular area of the colour and matte image is recorded where there is a picture to be matted in post production. This replays in exactly the same way as in the computer-generated case described above.

In accordance with a second aspect of the invention, the recorded video signal comprises a luma component, one or more colour components, and a matte in synchronism on a single or common record medium. Preferably the signal is recorded with luma, colour, and matte samples interleaved. Such an arrangement is particularly suited for use with the memory reduction system described above, for use with solid-state stores.

In a preferred system, in order to provide the maximum

flexibility from maximum time storage of the least information to storage of the maximum amount of information we have adopted a 32- bit word format for memory. This allows a simple 32-bit memory array to be used for a variety of recording formats:-

(i) The 32-bit word may contain two luma samples and two chroma samples. This is the standard recording format where colour signals sampled in the ratio 4:2:2 (luma:colour difference 1:colour difference 2) are required. One second of this signal will occupy about 20 MegaBytes of memory if full frame images are recorded.

(ii) The 32-bit word may contain 4 luma samples only, providing twice the duration of recording to method (i) but recording in black and white only. This alternatively nay be used for pure matte recording. One second of this signal will occupy about 10

MegaBytes of memory if full frame images are recorded.

(iii) The 32-bit word may contain 1 luma sample, 1 chroma sample and 1 matte sample for parallel recording of matte and colour.

This leaves one 8-bit byte free. This may be used for recording an additional 2-bits of accuracy in each of the samples, providing 10-bit accuracy of samples (not currently available in commercial recording machines, but allowed for in the digital TV standards). The last two bits of the 32-bit word may then be used for additional recording of information in parallel with the TV signals, providing approximately 2.5 MegaBytes per second of channel capacity. This can record for instance (a) at least 10 channels of digital audio, and/or (b) information regarding the orientation and/or location of the camera.

In accordance with a third aspect of the invention, a video signal generated by a camera is digitally recorded, and data concerning the orientation and/or location of the camera are recorded with the video signal. Preferably the data comprises samples which are interleaved with the video samples. The additional data can subsequently be analysed by a computer, and a computer scene created, the viewpoint of which moves according to the original camera motion. This allows the camera images to be combined with the computer images, for example using matte overlay, to create an illusion of the real image being in the same scene as a computer background.

The invention is defined with more precision in the appended claims to which reference should be made.

The invention will now be described by way of example with reference to the accompanying drawings, in which:-

Figure 1 is a diagram illustrating the storage requirements of an object stored without a matte on a static background;

Figure 2 illustrates the storage of the video signal in memory showing the interrelation of the frames, and the interrelation of the samples within a frame;

Figure 3 illustrates the reconstruction of frames from the stored data;

Figure 4 is a block circuit diagram of a basic circuit for reconstructing an image from stored information;

Figure 5 is an alternative reconstruction circuit using an on/off matte signal;

Figure 6 illustrates an object stored together with its matte;

Figure 7 is a diagram similar to Figure 2 for a system in which matte data is stored together with additional data;

Figure 8 illustrates the reconstruction of a full frame image using a matte;

Figure 9 further illustrates the use of the image and matte from Figure 8 to produce a composite image;

Figure 10 is a block circuit diagram similar to Figure 5 for a system using a matte; Figure 11 is a detail showing a modification to Figure 10 with the inclusion of a switch to black in the video path;

Figure 12 is a more detailed circuit illustrative of how the data is read from memory;

Figure 13 is a block circuit diagram illustrating the derivation of the bounding rectangle from an incoming matte;

Figure 14 is a double buffered input circuit for buffering the input while determining the bounding rectangle;

Figure 15 shows a camera which will also produce orientation and/or location data;

Figure 16 shows an encoding system for position and/or orientation data;

Figure 17 shows a decoding system for position and/or

orientation data;

Figure 18 shows a system for generating computer images to match camera orientation; and

Figure 19 shows a comparison of the various stages of image and matte generation for two different camera orientations.

Referring to Figure 1 there are shown two frames namely frames 1 and 2. In frame 1 in the lower part of the figure an aircraft is shown which is to be recorded. A bounding rectangle is shown surrounding the aircraft. This is defined by the location x₁, y₁ of the top lefthand corner of the rectangle, together with the horizontal width w₁ and the height h₁ of the rectangle. The aircraft is shown in the full television raster.

Frame 2 shows the aircraft in a different position where the location and size of the bounding rectangle are defined by x₂, y₂, w₂ and h₂. In each case the background to the two frames is static. It is therefore possible to record the content of the frames in its entirety by storing the background picture, and the portions of the two frames within the bounding rectangles. As shown, the bounding rectangle is redefined for each frame and in this way the amount of storage required is minimised.

As shown in Figure 2, the part of the picture within the bounding rectangle is stored in memory. It will be seen that the area required for frame 1 is smaller than that required for frame 2. As shown in the detail of Figure 2, the samples are stored in Y-U-Y-V form using eight bits per sample.

In order to reconstruct the frames, the fixed background is required and the recorded frames 1 and 2 are used to replace corresponding parts of the background by switching between the recorded frames and the background at the edges of the bounding rectangle. This is illustrated in Figure 3.

A suitable circuit for such purposes is shown in Figure 4. A background frame store 10 holds the background picture. This is sufficient to accept one frame of video. A large memory array 12 is used to store the portions within the bounding rectangle for each of the succession of frames. The memory array will be as large as possible or practicable for any given application. A video switch 14 selects the output of the background frame store 10 or the memory array 12 in accordance with a control input which constitutes a switching signal identifying whether the displayed picture portion at any instant is within the bounding rectangle or not. The switching signal is derived from scan address circuitry 16 which receives synchronising information concerning the television signal derived by a known genlock circuit which receives the reference video and provides timing data to the scanning circuitry along with a pixel rate clock arranged to be in synchronisation with the incoming video and which is part of the genlock circuitry, and also receives information from a stored database 20 of rectangle sizes and memory locations. This correlates the values x,y,w and h for the bounding rectangles in each frame with the location in memory array 12. By comparison of these values the scan address circuitry 16 can generate the switching signal. Scan address circuitry 18 is used to address the background frame store 10.

In an alternative arrangement shown in Figure 5 use is made of an on/off matte signal. For this purpose the switching signal from scan address circuitry 16 is applied as a control input to a switch 22 which selects as its input one of two signals respectively indicative of transparent and opaque. The appropriate one of these signals is then applied to a mixer 24, replacing the video switch 14 of Figure 4, and the output of which constitutes the combined image output.

Figure 6 illustrates the matte associated with each of the aircraft illustrated in Figure 1. It will be appreciated here that the switching between background and stored information takes place not at the boundary of the rectangle, but rather at the boundary defined by the edges of the matte. The matte is shown in the righthand side of Figure 6 for each of the frames shown on the lefthand side. The data storage structure for a system

incorporating a matte may be as shown in Figure 7, where the successive stored bytes contain Y-U-M-X-Y-V-M-X. In this case U and V are the blue and red colour difference signals of a

conventional Y,U,V signal, and M is the matte signal. The X byte may be used for different purposes as has already been noted. In one example it may contain an additional two bits of data associated with each of the three preceding bytes respectively, together with a further two bits. The further two bits may contain audio data or control information. This control information may, for example, include data concerning the location and/or orientation of the camera which was used to produce the image.

Using the arrangement of Figures 5 to 7 to reconstruct a full frame image, as shown in Figure 8, involves generating full frames of the foreground image, that is to say including the aircraft, and of the matte. Thus as shown in Figure 8, a full frame foreground image is generated comprising the data within the bounding rectangle surrounded on its outside by an arbitrary surround colour.

Likewise, a full frame matte is generated which the area outside the bounding rectangle represents transparent. This is achieved by a switching operation at the edge of the matte bounding rectangle. This operation is undertaken for each frame as shown in Figure 8.

The resultant full frame matte is then used to combine the foreground image with a stored background image. This is

illustrated in Figure 9 where the background video has the aircraft superposed upon it using the matte as a control input. Matting in this way is in itself a well-known operation. However, the circuitry shown in Figure 10 can be used for this purpose.

Figure 10 is similar to Figure 5 except that the selector switch 22 now receives either a signal representing "transparent" or stored matte information from the memory array 12. Typically a value of '0' for the matte information represents transparency, whilst a value of 255 represents opacity. Values in between represent varying degrees of translucency. It should be noted that the background frame store 10 could be a stored single frame but could be constituted by other sources such as a further output from a large memory array, or an incoming live video signal. In the modification shown in Figure 11 a further selector switch 26 is included in the path of the stored colour video information between the memory array 12 and the mixer 24. The selector switch 24 has a signal representing black at its other input. This may be used to feed an optional monitor 28 and has the advantage that a "clean" view of the stored foreground information appears on the monitor. The control for the switch 24 is in parallel with that for the switch 22.

Figure 12 shows in the top part of the figure in somewhat more detail the reading of the data from the large memory array 12.

Thirty two bits are read at one time and are held in a four byte latch 30. Output latches 34a to 34d are provided of which three generate a 10 bit output respectively of luminance, chrominance and the matte information, and the fourth generates a 2 bit output of additional data. For this purpose buffer circuits 36 are included as shown.

Between the four byte latch 30 and the 10 bit latches are ten latches or buffers 38a-38j and three further 2-bit latches or buffers 40a-40c.

The latch 30 receives the data from the memory array 12. The data may be in one of the 5 formats shown in the lower half of the diagram.

Format 1 shows where luma only is recorded. In this case four samples are clocked at 1/4 of the pixel rate. A control mechanism (not shown) then enables each of 4 buffers in turn to enable each Y byte in turn into the Y latch. The latches enabled are 38,a,b,c,d in turn on each pixel clock. After 38d is enabled another 32-bit sample is latched in 30 for the next four samples.

Format 2 shows a similar format where Matte only is recorded. In this case buffers 38g,h,i,j are enabled to provide the data to the matte latch on the right.

Format 3 shows where a colour TV signal only is recorded. Samples are now clocked from RAM at 1/2 the pixel rate. The buffers enabled are 38a and 38e to latch the first Y and C samples into the output latches, then 38c and 38f to enable the second sample. After this another 32-bit sample is latched from large memory array and the cycle repeats.

Format 4 shows where colour video and a matte is stored, but with no additional information. X indicates an unused byte.

Memory is clocked out at full pixel rate, and buffers 38a,e,i enabled to let the data through to the output latches on each cycle.

Format 5 shows where the unused byte X is replaced by

additional data P. P represents additional precision to format 4, so that additional buffers 40a,b,c are enabled to let the two additional precision bits into each 10-bit store. Also the final two bits of additional data are latched into the A output

buffer 34d.

It is assumed that the buffers 38,40 are arranged so that when not enabled, a 0 value is provided to the subsequent circuitry to ensure that the output latches remain at zero where there is no recorded information.

The block at the base of the Figure shows the composition of the additional byte having 2 bits each of luma, colour, matte and additional information.

The derivation of the bounding rectangle from an .meaning matte is illustrated in Figure 13. The top half of the figure shows the derivation of the y coordinates of the top and bottom of the rectangle. The matte signal is received at an input 50 and applied to a comparator or slicing circuit 52 which provides a high output when the input signal is above the slicing level. This thus provides a high output in the region of the matte and a low output outside it. This signal is applied to a gate 54 which thus passes clock pulses from a pixel rate clock 56 when the matte is present. To determine the y coordinates at the top of the figure, these pulses are applied to a D-type flipflop or latch circuit 58. This latch is reset by frame sync pulses which are also fed to a line counter 60. The value of the line counter is applied to top and bottom latches 62 and 64. The bottom latch is preloaded with zero and is loaded with the counter value on receipt of the first of the pulses from gate 54. The top latch 62 receives only the first pulse from gate 54 as once set, D-type 58 remains set for the remainder of the frame. Thus latch 62 retains latched the number of the first line which contained matte information above the slicing level. Latch 64 however latches for every line where there is matte, and therefore the last value in here will be the last line on which matte appears. Thus latch 62 contains the first line and latch 64 contains the last line.

At the bottom of the figure somewhat similar circuitry is shown in respect of the other two sides of the rectangle. The output of gate 54 is applied through gates 70,72 to a left latch 74 and a right latch 76. A pixel counter 80 is incremented pixel by pixel and reset at the end of each line. A comparator 82 compares the pixel count from counter 80 with the value in the left latch 74 and provides an output which is high when the pixel count A is less than the value B in the latch so as to open gate 70. This causes the pixel count to be loaded into the latch.

Conversely the comparator 84 compares the pixel count from counter 80 with the contents of the right latch 76, and provides an output to gate 72 which is high when A is greater than B.

The left and right latch are preloaded respectively with the maximum pixel count and with the zeros and with the operation as described will ensure that the latches 74 and 76 contain at the end of each frame the value appropriate for the left and right edges of the bounding rectangle.

Thus, at the end of the frame the four latches contain the extreme limits of the matte to be recorded from which the desired x,y can be read off, and w and h calculated. These are stored for future reconstruction of the image.

Figure 14 shows how the information is applied to the large memory array 12. A camera 100 views a foreground scene containing say an actor in front of a blue back drop as in itself wellknown. From the camera the video signal is applied to an analogue-to-digital converter 102, and conventional matte deriving equipment 104, such as that known under the trade mark ULTIMATTE is used to derive a matte signal which is converted to digital form by an analogue-to-digital converter 106. There are then four frame stores 108,110,112,114. These are used in two pairs, with stores 108,110 being concerned with frames A and stores 112,114 being concerned with the alternate frames B. Stores 108 and 112 contain video information from ADC 102, and stores 110 and 114 contain matte information from ADC 106. The output of ADC 106 is also applied over a line 116 to the bounding box deriving circuit as shown in Figure 13. The outputs of the four frame stores are then selectively applied to the memory array 12.

The two sets of frame stores A and B are required so that one set can be written to while the other set is being read from. For this purpose two switching circuits 118 and 120 are included which operate alternately. Switch 118 receives synchronising signals from a Genlock signal source after appropriate modification in a circuit 122. Switch 120 receives the output of a circuit 124 which calculates the appropriate addresses having regard to the size of the box for the immediately preceding frame, as calculated by the circuit of Figure 13. The circuit 124 thus ensures that the parts of the frames being stored, that is to say the parts within the bounding boxes, are stored sequentially in the memory array 12 without wastage of space.

It is seen from the above that the picture portion containing the moving image is respecified on each frame. The frame in this case may be a single field or else two interlaced fields forming the single picture of a video display. In appropriate circumstances it may be sufficient to respecify the box area once every few frames rather than on every successive frame, but sufficiently frequently to ensure that there is no significant wasted storage area.

The recorded video signal which is held in the memory array 12 comprises a conventional colour signal including luminance and chrominance components in synchronism with a matte. The video signal is preferably stored in the form of YUV components and the matte may be stored, as has been described with reference to Figure 7, by interleaving the matte samples with the video signal samples. Where this is done on a four byte cycle the fourth byte may

conveniently be used for other purposes which may include inter alia the storage of data concerned with the orientation and/or location of the camera 100 which was used to capture the foreground scene. If orientation/location data is included, the background scene with which the moving foreground is combined such as by use of the circuit of Figure 10 may itself be computer generated. The orientation and/location of the apparent picture source in this computer generation may then be determined by the additional data held in the memory array and relating to the original foreground camera 100.

Orientation and/or location data is derived from a camera of the type shown in Figure 15. A mobile position sensor 130 mounted on a camera 132 produces signals representing the camera's position in the x,y and z directions with respect to a remotely mounted fixed sensor 134.

The camera can rotate about a vertical axis 136 and a

horizontal axis 138. Potentiometers 140 and 142 mounted on the vertical and horizontal axes respectively produce signals relating to the azimuth and elevation of the camera. A further

potentiometer 144 attached to a zoom control of the camera produces a signal representing the amount of optical zoom which is being used.

If only location data is required then only the position sensors 130 and 134 are used. Similarly, only the potentiometers 140,142,144 are used if only orientation data is required.

Data from the position sensors 130 and 134 and/or the

potentiometers 140,142 and 144 is encoded using the system shown in Figure 16. A to D converters 146 receive signals from the potentiometers 140,142 and 144 and supply converted digital signals to a CPU 148. This outputs a serial data stream containing the orientation data which nay be recorded by a video tape recorder (VTR), or, alternatively may be encoded into an audio signal via a modem 150 for recording the data on an audio track of a video tape.

An alternative arrangement is shown in the lower half of the Figure. In this, the position sensor 130 has six degrees of freedom which will include elevation and azimuth. The fixed sensor 134 is a 3-dimensional space tracker which outputs serial data directly to the CPU 148. If the sensor 130 only detects x,y,z position data then only this will be output to the CPU which can use it, in combination with, or as an alternative to, the orientation data.

If the position and orientation parameters are calculated to an accuracy of 20 bits at 30 frames per second then this gives a total of 4200 bits per second. This can be transmitted serially at 9600 baud by the CPU 148 and a modem allows this data rate to be easily stored on an audio track of a video tape.

If a video tape is not used to store position and/or

orientation data then it can be included in the additional

information channel of the RAM recorder. It will be appreciated that the concept of recording position and orientation parameters in synchronism with the video signal is applicable to analogue as well as digital systems.

Figure 17 shows a decoding arrangement for the position and/or orientation data. Orientation data from the VTR (via a modem if contained in the audio track) is received by a CPU 152. Position data is also received from the VTR by the CPU 152. The CPU computes a list of orientations for each frame of the video tape and stores these in a RAM block 154. If the orientation and/or location data is stored in the additional information channel of the RAM recorder then the orientation list may be computed directly from this.

Computer images to match camera orientation are generated using a system as shown in Figure 18. A 3D rendering machine 156 receives camera orientation and/or position data and data from a stored 3D model 158 of the object to be overlaid. The stored object data includes data relating to the position of the object to be overlaid in the final image. The 3D rendering machine 156 generates a video image of the object which is output to an RGB frame buffer 160, and a matte signal which is output to a further frame buffer 162. It also generates the bounding rectangle information. These three signals are then available for

recording. Video data relating to the background is stored in parallel with the camera orientation and/or position data.

Figure 19 shows how the object is matted into a background scene at frams 0 and frame 100. Figures 19(a) and (b) show the different position of the camera 132 for frames 0 and 100

respectively and the position 164 of the object which is to be overlaid (the bounding rectangle). Figure 19(c) shows the scenes as viewed by the camera at frames 1 to 100, 19(d) shows the object to be overlaid as generated by the computer within the bounding rectangles 164. The corresponding computer generated mattes are shown in Figure 19(e) and the final composed images in Figure 19(f).

In the system of Figures 18 and 19, the computer may of course generate the background scene and a matte instead of the object and a matte.

When the computer generates a foreground object, there are many methods of calculating the colour of each pixel. These often produce a colour for a pixel which depends on contributions from various details of the object smaller than a pixel according to their proportional coverage of the pixel area. The matte for all such pixels which are ultimately completely covered by the object being rendered is thus opaque.

When a pixel which intersects the edge of an object is rendered, a colour contribution from the object is calculated according to the proportion of the pixel which is covered by the object, the rest of the attribution is to be from the background. Because the background is not known and the final matting process is to be done later using the matte value as the constant in a linear interpolation equation, it has been found necessary to take the colour calculated for the pixel with no background and divide it by the (fractional) matte value (which represents area covered) prior to storing it as the colour value for a pixel. (This is required for each of the colour components of the image at this pixel).

The matte value stored is the proportion of the pixel area covered by the object. This allows the matting process to be done later to generate the correct colour for such an edge pixel when matted over an arbitrary background.

Where the image is rendered in Y, U and V components with the U and V components at a lower resolution than the Y component, the average value of the matte values for the several Y samples which correspond to an individual colour sample is used as the divisor factor. Any slight inaccuracy in the final matting of the object over the background is confined to the colour signals to which the eye is less sensitive. The luma values are correctly calculated. The system has been described on a frame by frame basis without interlace. In practice, interlace will often be used and the bounding box will be determined on a field by field basis for live video. This will lead to a corresponding reduction of the amount of storage required for the system of Figure 14. Thus images may be stored for a complete frame or for each field.

All the methods described herein can be used in an RGB system. The only differences will be in the manner in which bytes are stored which, in the above description, are specific to YUV.

Claims

1. A method of storing a digital video signal, comprising the steps of repeatedly determining a portion of the picture area which contains moving information, and storing only the portion of the video signal which corresponds to said portion of the picture area.

2. A method according to claim 1, in which the said determination is made on a frame-by-frame basis.

3. A method according to claim 1 or 2, in which the said portion of the picture area is defined as a rectangular picture portion.

4. A method according to claim 1, 2 or 3, including storing a matte signal which corresponds to the moving information in the picture portion, only the portion of the matte signal being stored which corresponds to the said portion of the picture area.

5. A method according to claim 4, in which the matte is recorded in synchronism with the recorded video signal on a common record medium.

6. A method according to claim 5, in which the samples of the stored video signal and the matte are interleaved.

7. Apparatus for storing a digital video signal, comprising means for repeatedly determining a portion of the picture area represented by the video signal which contains moving information, and means for storing only the portion of the video signal which corresponds to said portion of the picture area.

8. Apparatus according to claim 7, in which the determining means operates on a frame-by-frame basis.

9. Apparatus according to claim 7 or 8, in which the determining means defines the said portion of the picture area as a rectangular picture portion.

10. Apparatus according to claim 7, 8 or 9, including means for storing a matte signal which corresponds to the moving information in the picture portion, only the portion of the matte signal being store which corresponds to the said portion of the picture area.

11. Apparatus according to claim 10, in which the matte is recorded in synchronism with the recorded video signal on a common record medium.

12. Apparatus according to claim 11, in which the samples of the stored video signal and the matte are interleaved.

13. A method of recording a digital video signal together with a matte, the video signal comprising luminance and dominance components, and the method including the step of recording the luminance and chrominance components and the matte insynchronism on a common record medium.

14. A method according to claim 13, in which the luminance, chrominance and matte samples are interleaved.

15. A method according to claim 14, in which the data is stored with successive samples of the form Y-U-M-Y-V-M, where Y are luminance samples, U and V are chrominance samples, and M are matte samples.

16. A method according to claim 14, in which the data is stored with successive bytes of the form Y-U-M-X-Y-V-M-X, where Y are luminance bytes, U and V are chrominance bytes, M are matte bytes, and X are additional bytes.

17. A method according to claim 16, in which the additional bytes contain further bits of definition for one or more of the Y,U,V and M bytes.

18. A method according to claim 16 or 17, in which the additional bytes contain data indicative of the location and/or orientation of the camera used to record the scene constituted by the video signal.

19. Apparatus for recording a digital video signal together with a matte, the video signal comprising luminance and chrominance components, and the apparatus including means for recording the luminance and chrominance components and the matte in synchronism on a common record medium.

20. Apparatus according to claim 19, in which the record medium comprises a solid-state store.

21. Apparatus according to claim 19 or 20, in which the luminance, chrominance and matte samples are stored in interleaved form.

22. Apparatus according to claim 21, in which the data is stored with successive samples of the form Y-U--M-Y-V-M, where Y are luminance samples, U and V are c±rαminance samples, and M are matte samples.

23. Apparatus according to claim 21, in which the data is stored with successive bytes of the form Y-U-M-X-Y-V-M-X, where Y are luminance bytes, U and V are chrominance bytes, M are matte bytes, and X are additional bytes.

24. Apparatus according to claim 23, in which the additional bytes contain further bits of definition for one or more of the Y,U,V and M bytes.

25. Apparatus according to claim 23 or 24, in which the additional bytes contain data indicative of the location and/or orientation of the camera used to record the scene constituted by the video signal.

26. A method of generating and recording a video signal, ccxrprising directing a camera towards a scene, recording a video signal derived from the camera output, and recording in synchronism with the recorded video signal information concerning the camera location and/or orientation.

27. A method according to claim 26, in which the camera is provided with means for detecting the location and/or orientation thereof and for generating signals dependent thereon.

28. A method according to claim 26 or 27, in which the location and/or orientation information is recorded in interleaved form in relation to the recorded video signal.

29. A method according to claim 26, 27 or 28, in which the video signal is a digital video signal.

30. Apparatus for generating and recording a video signal, comprising a camera to be directed towards a scene, means for deriving a video signal from the camera output, means for recording the video signal, means for detecting the location and/or

orientation of the camera and generating signals dependent thereon, and means for recording the location and/or orientation signals in synchronism with the recorded video signal.

31. Apparatus according to claim 30, in which the location and/or orientation signals are recorded in interleaved form in relation to the recorded video signal.

32. Apparatus for generating a video signal from recorded signals, comprising first store means for generating a recorded foreground signal, second store means for deriving recorded information related to the camera location and/or orientation when the foreground signal was recorded, a computer for generating a background signal, the computer receiving the recorded location and/or orientation information and adjusting the background signal which it generates in response thereto, and means for combining the foreground and background signals to provide a composite output signal.

33. A method for computing a video signal representing at least one object together with an associated matte, the method comprising the steps of computing, for at least each pixel intersected by an object, a colour value substantially equal to the colour of the object divided by a value derived from the proportional area of that pixel covered by the object, computing an associated matte value from the proportional area of that pixel covered by the object, and storing a video signal including the computed colour data for each intersected pixel together with the associated matte value.

34. A method according to claim 33 in which colour values are computed for each of the colour components of the image to be stored.

35. A method according to claim 33 or 34, in which the colour values are RGB values.

36. Apparatus for computing a video signal representing at least one object together with an associated matte, the apparatus comprising means for computing, for at least each pixel intersected by an object, a colour value substantially equal to the colour of the object divided by a value derived from the proportional area of that pixel covered by the object, means for computing an associated matte value from the proportional area of that pixel covered by the object, and storage means for storing a video signal including the computed colour data for each intersecting pixel together with the associated matte valve.