WO2007132471A1 - Tracking mouse locally and microsoft windows connectivity algorithm - Google Patents

Tracking mouse locally and microsoft windows connectivity algorithm Download PDF

Info

Publication number
WO2007132471A1
WO2007132471A1 PCT/IN2006/000163 IN2006000163W WO2007132471A1 WO 2007132471 A1 WO2007132471 A1 WO 2007132471A1 IN 2006000163 W IN2006000163 W IN 2006000163W WO 2007132471 A1 WO2007132471 A1 WO 2007132471A1
Authority
WO
WIPO (PCT)
Prior art keywords
anyone
server
mouse
remote
connectivity
Prior art date
Application number
PCT/IN2006/000163
Other languages
French (fr)
Inventor
Alok Singh
Ashok Zutshi
Yuvaraj Galada
Vinodkumar Gopinathan
Saleem Taha Mahin
Mukul Chitrao
Padmalakshmi Shankar
Selva Kumaran Rajan
Kumaran Meenatchi Sundaram
Johnson Adaikal Raj
Gowrisankar Radhakrishnan
Shuveb Hussain
Charu Latha Prasanna
Siva Rama Krishna Obul Reddy
Ramaseshu Krishnamurthy
Thalesh Ranganathan
Renganathan Muthukrishnan
Rajaraman Krishnan
Nisheeth Mishra
Original Assignee
Alok Singh
Ashok Zutshi
Yuvaraj Galada
Vinodkumar Gopinathan
Saleem Taha Mahin
Mukul Chitrao
Padmalakshmi Shankar
Selva Kumaran Rajan
Kumaran Meenatchi Sundaram
Johnson Adaikal Raj
Gowrisankar Radhakrishnan
Shuveb Hussain
Charu Latha Prasanna
Siva Rama Krishna Obul Reddy
Ramaseshu Krishnamurthy
Thalesh Ranganathan
Renganathan Muthukrishnan
Rajaraman Krishnan
Nisheeth Mishra
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alok Singh, Ashok Zutshi, Yuvaraj Galada, Vinodkumar Gopinathan, Saleem Taha Mahin, Mukul Chitrao, Padmalakshmi Shankar, Selva Kumaran Rajan, Kumaran Meenatchi Sundaram, Johnson Adaikal Raj, Gowrisankar Radhakrishnan, Shuveb Hussain, Charu Latha Prasanna, Siva Rama Krishna Obul Reddy, Ramaseshu Krishnamurthy, Thalesh Ranganathan, Renganathan Muthukrishnan, Rajaraman Krishnan, Nisheeth Mishra filed Critical Alok Singh
Priority to PCT/IN2006/000163 priority Critical patent/WO2007132471A1/en
Publication of WO2007132471A1 publication Critical patent/WO2007132471A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/363—Graphics controllers

Definitions

  • the present invention relates to an integrated convergence platform for voice, video and data applications including computing, communication and entertainment applications adapted for local mouse implementation facility at the user's access terminal / device involving embedded system having very constrained resources such as small processor and memory.
  • the local mouse implementation facility is adapted to provide a combination of VNC technology involving the integrated convergence platform and graphic operation technology, which would enable the user to not only implement and emulate the server screen at the user's access terminal by decompressing and display of the screen but also enable carrying our graphic operation.
  • the invention is also directed to windows desktop connectivity for such embedded systems having constrained resources and with or without operating system.
  • the invention would favour wide scale and user friendly application and use of integrity convergence platform including a local mouse implemented facility and also favour simple and ready connectivity of windows desktop from embedded systems without dependencies on high load consuming and complex X-servers.
  • a simple & single platform providing (i)means for computational application ;(ii)means for entertainment applications; and(iii) means for voice and data applications; plurality of customized user access terminals/devices operatively connected to said common platform for said desired triple play applications.
  • the above integrated technology platform rests on the server and provides access to the user for said computing or media applications. Importantly, to facilitate the management of the plurality of the user access devices a device management console is available.
  • the said user access terminals /devices are free of any moving components and/or heat generating units.
  • the integrated access system platform comprising of: plurality of user access terminal/devices in a server centric Computing environment; a LAN and/or WAN operating connection to server and a device management console.
  • the server cluster used in the system can be selectively comprised of Content servers and Computing Servers, or the functionality of the cluster could be provided by a single server.
  • client management software is also provided for managing multiple simplified user terminals / devices in the system.
  • the software is embedded in the hardware to impart desired operational capability to the hardware of the simplified convergence platform of the invention.
  • the convergence platform imparts simplicity and user friendly characteristics to the operation / use of triple play gadgets / components by way of providing for the first time very simple appliance like attributes to the access terminal/devices in the system of the invention.
  • appliance like attributes of the access terminals / devices would enable deployment of multiple access terminal / devices in hundred or thousands with very simple to manage network set up.
  • the system is adapted such that there is no additional burden or complexity of managing typical personal computer for individual work station.
  • the above problem of mouse implementation in turn affected the application and integrities of the simplified convergence platform with multiple access terminal involving PC with constrained resources.
  • the problem as discussed above is basically because the system of the convergence platform is a server based computing product. Importantly, it is found that though the server supported the local mouse, the packet format did not support the user terminal requirements and resulted in the above loss of control over the screen by the user due to delay in the movement of the mouse and its actual update on the screen based on server input.
  • the application have a communication protocol that interacts with windows server and uses X-server on client's side for providing input and output.
  • X-server on client's side for providing input and output.
  • Such existing applications work well for large processor architecture like normal X-86 based platform, since such platforms have enough resources in terms of processor's speed and memory to support a heavy resource consuming application like X-server.
  • X-servers are thus required by existing applications for input and output which is found to be lot taxing for embedded system with limited resources to support X-server which require lot of memory and processor's cycles.
  • Another object of the present invention is directed to providing the local mouse implementation as a combination of VNC technology and graphic operation technology, which would not only favour the user terminal / PC in embedded systems to emulate the server screen by de-compressing and displaying of the screen but also favour graphic operation.
  • Yet another object of the present invention is directed to local mouse implementation as a combination of VNC technology and the graphic operation technology.
  • a further object of the present invention is directed to a system where the mouse updates from the server and delay involved between the movement of the mouse by the user and the actual update of the screen could be avoided and the mouse movement synchronized with respect to the hand and the eyes of the user.
  • Yet further object of the invention is directed to achieving the mouse movement synchronized with the users movement of the mouse even in case of heavy flash or screen update and in environments where the traffic in the network is heavy and therefore take care of the problems of the user losing control over the screen due to delay action of mouse movement.
  • Yet further object of the present invention is directed to a server based computing product wherein the server is adapted to support local mouse movement and corresponding action.
  • a further object of the present invention is directed to capturing and drawing mouse movement locally and thereby solving the bottleneck of the mouse response being dependent of the network traffic and the server.
  • a further object of the present invention is directed to simple and cost-effective windows desktop connectivity from an embedded system having very constraint resources such as small processor and memory, which would specifically remove the dependency of X-server and be very useful in embedded environment with constant resources where it is taxing to run heavy application like X-server.
  • Yet another object of the present invention is directed to a windows desktop connectivity from an embedded system having very constraint resources, small processor and small memory, which would enable connecting embedded devices to support windows desktop connectivity thus saving cost on additional resources such as processing capabilities, memory and the like.
  • Yet further object of the present invention is directed to windows desktop connectivity from an embedded system, which can be used as an architectural model for thin client having minimal processing capabilities.
  • an integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications comprising:
  • plurality of user access terminal/devices having respective network processor with constrained resources with or without additional network processors of varied capabilities in a server centric Computing environment;
  • server means comprising a network server and a media server ;
  • a local mouse implementation facility comprising said media processor adapted to store locally the initial mouse shape received form the server; said network processor adapted to capture the coordinates whenever the user moves the mouse and to forward the coordinates to the said media processor and the server; said media processor adapted to draw the mouse on the screen with respect to the x and y coordinates ;said server adapted to recognize any change in shape of the mouse and forward the pattern and mask to the to the network processor; the network processor further adapted to receive the same as a separate mouse packet and forward the data to the media processor; and said media processor adapted to recognize and handle the mouse packet and draw the packet on the screen as a graphics operation.
  • the network server is adapted to provide other screen updates as a regular VNC screen update and is decoded and displayed by the client/network processor. Also, the network server is adapted to support such as 2 byte mask information for communicating with said access terminal/devices having respective network processor with constrained resources. Also, the network server is adapted to strip the unused portion of the mouse area and forward only the modified area for mouse drawing operations to said access terminal/devices having respective network processor with constrained resources. Moreover, the network server is modified only to the extent of supporting the access terminal/devices having respective network processor with constrained resources.
  • the client/network processor is adapted to forward mouse packet to the server and also use the data within itself for drawing the mouse.
  • the client/network processor is adapted to identify the mouse packet and the screen updates as two different packet types and handles it differently such that when the packet is corresponding to the mouse packet it does graphic operation and draws the mouse locally , otherwise , the screen update packet is drawn as in the original VNC protocol.
  • the integrated convergence platform with local mouse implementation facility it is possible to achieve server scalability for said access terminal/devices having respective network processor with constrained resources and also all other clients connected to the same server through additional network processors of varied capabilities.
  • the integrated convergence platform with local mouse implementation facility comprises said platform having embedded customized software for providing computing, entertainment and voice capabilities.
  • a device management console is provided for managing said plurality of user access terminals/devices.
  • the said user access terminals /devices are free of any moving components and said device operates menu based graphical user interface (GUI).
  • GUI graphical user interface
  • the said device connects to an operating system server also providing computing applications as one of its features.
  • the device comprises embedded software specifically developed to control and provide the said features and are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
  • Such devices are application class computer access terminals with multimedia capabilities.
  • the said server means can comprising Content servers and Computing Servers.
  • the integrated convergence platform with local mouse implementation facility comprises any or all of the following depending on the users' application requirement -
  • General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units.
  • the system can also be adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
  • the integrated convergence platform with local mouse implementation facility comprising the following hardware blocks -
  • Universal Asynchronous Transmitter and receiver means Serial Peripheral Interface; SPORT means; Peripheral Component Interconnect Bus means;
  • USB Controller Audio Coder-Decoder means
  • Video Digital to analog converter and / or VGA Controller Speaker/MIC means; Interface to Monitor and / or TV;
  • VoIP Voice over Internet Protocol
  • a computer readable information recording medium which records a program /algorithm for providing an integrated convergence platform for voice, video, and data applications including computing, communication and entertainment applications with local mouse implementation facility as discussed above.
  • a remote windows desktop connectivity from embedded systems with constrained resources comprising:
  • a remote desktop protocol operatively connecting to windows terminal server at one end and to a wrapper means at the other end ;
  • said wrapper means disposed there between said remote desktop protocol and a custom display hardware device having embedded graphical operations.
  • the remote desktop protocol is adapted to directly communicate with said hardware and comprises a key map wrapper for identifying the key strokes and converting the same to necessary format.
  • the remote windows desktop connectivity comprises a remote desktop protocol network layer adapted for interacting with Windows Terminal Server, a remote desktop protocol decoder means adapted to decode the data that passes to the wrapper means, said wrapper means provided as an interface between said display device and said decoder means.
  • the remote desktop protocol display device comprises a graphical Operations Core operatively connected to a Display Driver and a Device Cache said Display Driver in turn operatively connected to an output means whereby the core interacts with the Display Driver to pass the output to the appropriate output device.
  • the graphical operations core comprises all graphical operations including preferably memory blitting, text drawing.
  • the embedded system for remote connectivity can have no operating system or adapted to run on embedded Linux operating system.
  • the remote windows desktop connectivity can comprise plurality of said remote desktop protocols and remote user access desktop device comprising embedded customized software for providing computing, entertainment and voice capabilities.
  • a device management console for managing said plurality of user access terminals/devices.
  • the said user access terminals /devices are free of any moving components and the device operates menu based graphical user interface (GUI).
  • GUI graphical user interface
  • the said device connects to said operating system server also providing computing applications as one of its features.
  • device comprises embedded software specifically developed to control and provide the said features.
  • such devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
  • the devices are application class computer access terminals with multimedia capabilities.
  • the said windows terminal server means can comprising Content servers and Computing Servers.
  • the remote windows desktop connectivity can be included in systems comprising any or all of the following depending on the users' application requirement -
  • General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units.
  • the remote windows desktop connectivity including remote access protocol adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
  • the said remote windows desktop connectivity include remote access protocol with the following hardware blocks -
  • Peripheral Component Interconnect Bus means; USB Controller; Audio Coder-Decoder means; Video Digital to analog converter and / or VGA Controller;
  • Speaker/MIC means Interface to Monitor and / or TV; A power source,
  • VoIP Voice Over Internet Protocol
  • a computer readable information recording medium which records a program /algorithm for providing remote windows connectivity from an embedded platform using the said remote windows desktop connectivity as detailed above.
  • the above system of the invention involving the local mouse implementation facility would provide for a combination of VNC technology and graphic operation technology in thin client computing environment.
  • VNC technology and graphic operation technology in thin client computing environment.
  • the system architecture of the invention would enable achieving mouse movement generated by the user almost instantaneously without requiring the user to lose control over the screen temporarily.
  • the system would take care of constraints of thin client computing environments such as the integrated convergence platform for voice, video and data application and bring in the much required compatibility between the server and the local access /terminal and overcome the bottleneck of implementing the local mouse technology due to unsupported packet format of existing VNC technology.
  • the constraints of access terminal / PC with limited resources are found to have problems to process data due to memory limitation. This has been taken care of at the server by stripping the unused portion of the mouse area and sending only the modified area for the mouse drawing operation. Also the server side changes are adapted such that it does not affect other regular X-86 client's connected to it and only the user terminal / PC with limited resources have the effect of such change in the server configuration.
  • the windows desktop connectivity from an embedded system having constraint resources such as small processor and small memory have been achieved in an manner whereby it is possible to avoid the dependencies on X- server and make for the much desired effective use of embedded environment with constraint resources where it is taxing to run heavy applications like X-server. Using such system it would now be possible for the embedded devices to be able to support windows desktop connectivity, thus saving cost on additional resources such as processing capabilities, memory etc.
  • the above remote desktop protocol eliminates any known dependency on X- 11 and X-server.
  • the wrapper between the present RDP and a custom display hardware has been provided for such purpose in the system.
  • the graphical operations are directly embedded on disk play hardware.
  • the present RDP communicates directly to hardware and has a key map wrapper for identifying the keystrokes and converting the same to the necessary format.
  • the embedded devices would be able to support windows desktop connectivity thus saving cost on additional resources.
  • FIGURES Figure 1 Illustrates a multimedia capable server based computing environment provided by the convergence platform as in our co-pending PCT International application;
  • Figure 2 Illustrates a system block diagram describing the various high level system blocks as used in the system of our said co-pending application;
  • Figure 3 is a block diagram illustrating the mouse movement in a typical existing VNC protocol as per the existing system
  • Figure 4 is a block diagram illustrating the mouse movement involving the VNC technology and graphic operation technology in accordance with the present invention for use with the embedded systems in particular the systems as per figures 1 and 2 above;
  • Figure 5 is an illustration of the architecture diagram of remote desktop application with X- server of the existing art
  • Figure 6 is an illustration of architecture diagram of remote desktop application without X- server and involving the wrapper in accordance with the present invention.
  • Figure 7 is an illustration of the display device used in the system involving the remote desktop application with wrapper and without X-server in accordance with the invention.
  • FIG. 1 illustrates the provision of thin client computing environment in accordance with the system providing for the simple and effective operative connection to plurality of access terminal / devices and the triple play application area comprising content server [2], computing server [3] etc.
  • the simplified convergence platform provides for the much desired server centric computing environment by operative communication with plurality of access terminal / devices (AT) [6] and with the required servers for the triple play applications including the content server (CS) [2] and computing server (CS) [3].
  • the management of the plurality of the access terminal (AT) [6] is achieved by means of the device management console (DMC) [5].
  • DMC device management console
  • such device management console (DMC) [5] is simple and user friendly since the same is required to only manage very simplified access terminal (AT) [6] which do not use any moving parts / components or have any heat generating resources.
  • the operative connection of the convergence platform to the operative servers and the management console is attend through LAN / WAN environment.
  • the system basically comprises of General Purpose Processors (GPP) [7] to communicate over the network, handle tasks and manage the peripheral interfaces and a media processor (MP) [8] which is also a central processing unit adapted to handle media specific part of the platform.
  • GPP General Purpose Processors
  • MP media processor
  • the memory block (RAM) [9] indicates random access memory modules used by one or both the processors [8]. It is required for temporary calculation for running programs.
  • a flash means (FM) [10] which is basically a persistence storage area for software code that is also used along with optional provision of a smart card reader (SMR) [11] for high security environments.
  • the General Purpose Processor (GPP) [7] is a general purpose CPU (Central Processing Unit). Its main function is to communicate over the network, handle tasks and manage the many peripheral interfaces available to it. It may or may not be present in all configurations of the hardware platform described, In which case the Media Processor [8] takes over the general purpose processing.
  • GPP General Purpose Processor
  • the Media Processor (MP) [8] is the CPU that handles the media specific part of the platform and interfaces ' to output devices like TV, monitor [34], speaker [33], etc.
  • the access devices [6] in the present system when used in a server centric environment, have full multimedia capability unlike thin clients, allowing the use of the device as a media access client or as Video conferencing equipment.
  • the General Purpose Processor [7] is unavailable in certain configurations, it takes over its role as well.
  • the System Bus (SB) [13] is a set of wires that connects together various components with the CPU.
  • a Smart Cards Reader means (SCR) [11] is preferably provided to read authentication cards such as Credit Cards, ATM Cards, Identity ' Cards, Access Cards, etc. In high security environments, where typically a user needs to be identified with much more than a user name and password, this can be used to check if a user is actually producing a valid card.
  • the Smart Card Reader (SCR) [11] is basically a hardware device used in the system that is used to read information from Smart Cards [H].
  • the Simplified Convergence Platform design offers the following hardware blocks —
  • Ethernet MAC (EMAC) [14] or the EMAC [14] which is the Ethernet controller that is responsible for the hardware connectivity part of the Networking Subsystem.
  • Serial Peripheral Interface (SPI) [18] is a serial bus used to communicate with other SPI [18] capable devices. Since the media processor [8] has this capability, it is used to communicate between the General Purpose [7] and Media processors [8].
  • Serial port Means identified as (SPORT) [19] comprising a Serial port. In the system, it is used for inter- processor communications.
  • PCI Peripheral components Interconnect Bus
  • the component USB COntr [23] is the USB controller that helps communicate between the General Purpose Processor [7] and various USB devices.
  • the USB controller [23] controls devices such as the Mouse [28] , the Keyboard [27] , the Fingerprint scanner [29] used for user authentication and Web Camera [26] used for Video Conferencing.
  • the (PCI) expansion slot [24] is an optional block that can be present to accommodate commercially available hardware expansion slots like graphics cards or sound cards.
  • the Audio CODEC [31] is an Audio Coder-Decoder that helps with digital audio encoding and decoding. Internally, the system, like all computers deals with digital signals. But any devices like speakers [33] and VGA monitors [34] need analog signals.
  • the Video DAC [32] or Digital to Analog converter is one such device that converts the digital signals delivered by the Media processor [8] into analog signals that the VGA monitor can display.
  • the system further includes speakers/mic (SP) [33] which is a normal set of speakers and microphone used commonly in computers. Also monitors (MN) [34] comprise a regular PC monitor.
  • SP speakers/mic
  • MN monitors
  • the system is powered by a power supply (PS) [35] which is the unit that supplies DC power to the whole system.
  • PS power supply
  • FIG. 3 illustrates the mouse movement and the cause of delayed response from the server in the existing VNC protocol as per accompanying Figures 1 and 2.
  • the mouse movement is sent to the server (SV) and mouse update is also a part of the screen update.
  • the mouse update is from the server (SV) in such existing system there is delay involved between the movement of the mouse by the user and the actual update of the screen.
  • mouse movement is not synchronized with respect to the hand and eyes of the user (CL1/CL2)). This is more in case of heavy flash or screen updates where the traffic in the network is heavy which lead to further slowing of the mouse response in such conventional system.
  • the block at the client shows that the client (CLl) is sending the mouse packet to the server (SV) as well as using the data within itself for drawing the mouse. Only when there is a change in mouse shape, the server is adapted to send the new shape to the client as a separate packet.
  • the client (CLl) identifies the mouse packet and the screen update packet as to different packet type and handles it differently. If the packet is corresponding to the mouse packet, it does graphic operation and draws the mouse locally. Otherwise the screen update is drawn as in the original VNC protocol.
  • the initial mouse shape is received from the server and stored locally in the media processor.
  • the networks processor captures the coordinates.
  • the processors send the coordinates to both the media processor and the server.
  • the media processor draws the mouse on the screen with respect to the X and Y coordinates send by the network processor. If there is a mouse shape change, a server recognizes the change and sends the pattern and the mask to the client. The client (network processor) receives this as a separate mouse packet and gives the data to a media processor. The media processor recognizes and handles the mouse packet and draws the packet on titie screen as if it is graphic operation.
  • the other screen update comes from the server as a regular VNC screen updates and is decoded and displayed by the client. Since the mouse movement is recognized and handled on priority basis, the mouse movements are faster thus the technology solves the problem in delay in mouse response due to network traffic. This achieves a PC like experience for the user.
  • the X-Il cursor option sends the mask as 1 bit information.
  • packet type use in the access terminal PC supports only 2 byte information.
  • the access device / PC of the system which has constraints to process only 1,000 words of data for the mouse packet due to memory limitation. This is taken care at the server by stripping the unused portion of the mouse area and sending only the modified area for mouse drawing operation.
  • server side changes are done such that it does not affect the regular X-86 client connected to it but only the clients/ user terminal PC with constraint resources have the effect of this change.
  • FIG. 5 shows a conventional Remote Desktop Protocol (RDP) which communicates with X-server (XSV) with the help of X-11 (XIl) library.
  • X-Il serves as a wrapper between X-clients and X-server, isolating the underlying graphical implementation for application calls.
  • X-server initiates input / output devices and waits for request from X-clients.
  • the X-clients like remote desktop connects to the X-server (XSV) and starts sending graphical operations such as text, bitmaps, blitting etc.
  • X-server is also responsible for receiving inputs from input devices like mouse, keyboard etc. on behalf of the X-clients.
  • the remote desktop protocol avoiding the dependency on X-11(XIl) and X-server (XSV) in accordance with the present invention is illustrated in accompanying figure 6.
  • a wrapper (WR) between the remote desktop protocol and custom display hardware (CDH) has been provided. This enables the graphical operations directly embedded on the display hardware.
  • the remote desktop protocol involving the wrapper means of the invention directly communicates with hardware and has a key map wrapper for identifying the key strokes and converting the same to necessary format.
  • the remote desktop protocol network layer (RDPNL) is responsible for interacting with windows terminal server (WTSV) .
  • the remote desktop protocol decoder is adapted to decode the data that passes to wrapper, and interface between the architecture as clearly illustrated further in figure 6.
  • FIG. 7 illustrate by way of block diagram the display device architecture used in the system of the invention.
  • the graphical operation code contains all the graphical operations including memory blitting, text drawing to name a few.
  • This core (GOC) interacts with display driver (DD) to pass the output to the appropriate output device (OD).
  • DD display driver
  • OD appropriate output device

Abstract

The present invention relates to an integrated convergence platform for voice, video and data applications including computing, communication and entertainment applications adapted for local mouse implementation facility at the user's access terminal / device involving embedded system having very constrained resources such as small processor and memory. The invention is also directed to windows desktop connectivity for such embedded systems having constrained resources and with or without operating system

Description

TRACKING MOUSE LOCALLY AND MICROSOFT WINDOWS CONNECTIVITY
ALGORITHM
FIELD OF THE INVENTION
The present invention relates to an integrated convergence platform for voice, video and data applications including computing, communication and entertainment applications adapted for local mouse implementation facility at the user's access terminal / device involving embedded system having very constrained resources such as small processor and memory. Importantly, the local mouse implementation facility is adapted to provide a combination of VNC technology involving the integrated convergence platform and graphic operation technology, which would enable the user to not only implement and emulate the server screen at the user's access terminal by decompressing and display of the screen but also enable carrying our graphic operation. The invention is also directed to windows desktop connectivity for such embedded systems having constrained resources and with or without operating system. Importantly, the invention would favour wide scale and user friendly application and use of integrity convergence platform including a local mouse implemented facility and also favour simple and ready connectivity of windows desktop from embedded systems without dependencies on high load consuming and complex X-servers.
BACKGROUND ART
It is well-known that technology platforms are available for each of the computing- communication-entertainment such as mobile platforms for voice applications, integrated device platforms for voice and data application. However, mostly such devices or application presently available in the market are all personal computer based, Video on Demand Set-Top Boxes, Residential Gateways and Integrated Access Devices. Importantly, such presently available platforms such as the video on demand set top boxes are only capable of providing single function. Such devices which are application specific and personal computer based, are complex to use for mass adaptation. There are a few devices presently available in the market which are capable of handling any two of the above functions but a device that could provide simplicity in operation and use and yet facilitate triple play application selected from computing, communication and entertainment is highly desirable for the users of such system. In our co pending PCT International Application No. PCT/IN2005/000380 dated November 25, 2005 there disclosed an integrated simplified convergence platform for triple play applications including computing, communication and entertainment applications comprising
a simple & single platform providing (i)means for computational application ;(ii)means for entertainment applications; and(iii) means for voice and data applications; plurality of customized user access terminals/devices operatively connected to said common platform for said desired triple play applications.
The above integrated technology platform rests on the server and provides access to the user for said computing or media applications. Importantly, to facilitate the management of the plurality of the user access devices a device management console is available. The said user access terminals /devices are free of any moving components and/or heat generating units.
In our said co-pending application there is also disclosed the integrated access system platform comprising of: plurality of user access terminal/devices in a server centric Computing environment; a LAN and/or WAN operating connection to server and a device management console.
The server cluster used in the system can be selectively comprised of Content servers and Computing Servers, or the functionality of the cluster could be provided by a single server.
Further, client management software is also provided for managing multiple simplified user terminals / devices in the system. The software is embedded in the hardware to impart desired operational capability to the hardware of the simplified convergence platform of the invention.
It would be apparent from the above that the convergence platform imparts simplicity and user friendly characteristics to the operation / use of triple play gadgets / components by way of providing for the first time very simple appliance like attributes to the access terminal/devices in the system of the invention. Such appliance like attributes of the access terminals / devices would enable deployment of multiple access terminal / devices in hundred or thousands with very simple to manage network set up. Importantly, the system is adapted such that there is no additional burden or complexity of managing typical personal computer for individual work station.
While in our above discussed said co-pending PTC International application there is disclosed a simple and single technology platform to host variety of access devices selected from computing, communication and entertainment applications to facilitate multiple functions from common platforms and the server has been provided to operatively connect to such user's access terminal PCs to emulate the server screen at the user's access terminal, in the said system the mouse functionalities are essentially carried out in the server i.e. in the thick server and thin client architecture using VNC technology. In such system the mouse events are sent to server and the changes happen at the server end and then it is reflected at the client's user terminal. The mouse response is thus a part of the screen update to the client. Because of this, there has been a delay involved in moving the mouse and seeing the update on the screen. This delay depends on the network traffic and also on the server response.
It would be apparent from the above that the system involving the mouse update from the server that there is necessarily delay involved between the movement of the mouse by the user and the actual update of the screen. Thus the mouse movement is not synchronized with respect to the hand and the eyes of the user. Also when there is a heavy flash or screen update, the traffic in the network is heavy thus making the mouse response further slow. The mouse movement generated by the user in such system is thus almost ignored. It is thus very painful situation for the user as he loses control over the screen temporary.
It is therefore experienced that the above problem of mouse implementation in turn affected the application and integrities of the simplified convergence platform with multiple access terminal involving PC with constrained resources. The problem as discussed above is basically because the system of the convergence platform is a server based computing product. Importantly, it is found that though the server supported the local mouse, the packet format did not support the user terminal requirements and resulted in the above loss of control over the screen by the user due to delay in the movement of the mouse and its actual update on the screen based on server input. Moreover, apart from the above problems of mouse implementation it is found that windows desktop connectivity from embedded systems including in the simplified convergence platform in our said co-pending application, the windows desktop connectivity from such embedded system having very constrained resources such as small processor and small memory but involving dependencies on X-server required the running of the system with heavy application like X-server. Such required dependence of the system on the X-server in environments especially with embedded systems with constrained resources is found to be taxing and not desirable. There has thus been the need in the art to eliminate such dependencies on X-I l and X-server especially for achieving windows desktop connectivities from embedded system having limited and constrained resources.
In particular, in such presently available windows desktop connectivity from an embedded platform , the application have a communication protocol that interacts with windows server and uses X-server on client's side for providing input and output. Such existing applications work well for large processor architecture like normal X-86 based platform, since such platforms have enough resources in terms of processor's speed and memory to support a heavy resource consuming application like X-server. X-servers are thus required by existing applications for input and output which is found to be lot taxing for embedded system with limited resources to support X-server which require lot of memory and processor's cycles.
OBJECTS OF THE INVENTION
It is thus the basic object of the present invention to provide for simplified convergence platform adapted to facilitate triple play application / use including computing, communication and entertainment which would enable operative simple and user friendly access terminal having facilities for implementing the local mouse technology and thus avoid any afore discussed problems and complexities due to any possible loss of control over the screen in the server based mouse applications/response.
Another object of the present invention is directed to providing the local mouse implementation as a combination of VNC technology and graphic operation technology, which would not only favour the user terminal / PC in embedded systems to emulate the server screen by de-compressing and displaying of the screen but also favour graphic operation.
Yet another object of the present invention is directed to local mouse implementation as a combination of VNC technology and the graphic operation technology.
A further object of the present invention is directed to a system where the mouse updates from the server and delay involved between the movement of the mouse by the user and the actual update of the screen could be avoided and the mouse movement synchronized with respect to the hand and the eyes of the user.
Yet further object of the invention is directed to achieving the mouse movement synchronized with the users movement of the mouse even in case of heavy flash or screen update and in environments where the traffic in the network is heavy and therefore take care of the problems of the user losing control over the screen due to delay action of mouse movement. Yet further object of the present invention is directed to a server based computing product wherein the server is adapted to support local mouse movement and corresponding action.
A further object of the present invention is directed to capturing and drawing mouse movement locally and thereby solving the bottleneck of the mouse response being dependent of the network traffic and the server.
A further object of the present invention is directed to simple and cost-effective windows desktop connectivity from an embedded system having very constraint resources such as small processor and memory, which would specifically remove the dependency of X-server and be very useful in embedded environment with constant resources where it is taxing to run heavy application like X-server.
Yet another object of the present invention is directed to a windows desktop connectivity from an embedded system having very constraint resources, small processor and small memory, which would enable connecting embedded devices to support windows desktop connectivity thus saving cost on additional resources such as processing capabilities, memory and the like.
Yet further object of the present invention is directed to windows desktop connectivity from an embedded system, which can be used as an architectural model for thin client having minimal processing capabilities.
SUMMARY OF THE INVENTION Thus according to the basic aspect of the present invention there is provided an integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications comprising:
plurality of user access terminal/devices having respective network processor with constrained resources with or without additional network processors of varied capabilities in a server centric Computing environment;
a LAN and/or WAN operating connection to server means comprising a network server and a media server ;
a local mouse implementation facility comprising said media processor adapted to store locally the initial mouse shape received form the server; said network processor adapted to capture the coordinates whenever the user moves the mouse and to forward the coordinates to the said media processor and the server; said media processor adapted to draw the mouse on the screen with respect to the x and y coordinates ;said server adapted to recognize any change in shape of the mouse and forward the pattern and mask to the to the network processor; the network processor further adapted to receive the same as a separate mouse packet and forward the data to the media processor; and said media processor adapted to recognize and handle the mouse packet and draw the packet on the screen as a graphics operation. In the above integrated convergence platform with local mouse implementation facility the network server is adapted to provide other screen updates as a regular VNC screen update and is decoded and displayed by the client/network processor. Also, the network server is adapted to support such as 2 byte mask information for communicating with said access terminal/devices having respective network processor with constrained resources. Also, the network server is adapted to strip the unused portion of the mouse area and forward only the modified area for mouse drawing operations to said access terminal/devices having respective network processor with constrained resources. Moreover, the network server is modified only to the extent of supporting the access terminal/devices having respective network processor with constrained resources.
According to further aspect in the integrated convergence platform with local mouse implementation facility the client/network processor is adapted to forward mouse packet to the server and also use the data within itself for drawing the mouse. Importantly also, the client/network processor is adapted to identify the mouse packet and the screen updates as two different packet types and handles it differently such that when the packet is corresponding to the mouse packet it does graphic operation and draws the mouse locally , otherwise , the screen update packet is drawn as in the original VNC protocol.
Advantageously, in the integrated convergence platform with local mouse implementation facility it is possible to achieve server scalability for said access terminal/devices having respective network processor with constrained resources and also all other clients connected to the same server through additional network processors of varied capabilities. The integrated convergence platform with local mouse implementation facility comprises said platform having embedded customized software for providing computing, entertainment and voice capabilities. Preferably, a device management console is provided for managing said plurality of user access terminals/devices.
The said user access terminals /devices are free of any moving components and said device operates menu based graphical user interface (GUI). The said device connects to an operating system server also providing computing applications as one of its features. The device comprises embedded software specifically developed to control and provide the said features and are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration. Such devices are application class computer access terminals with multimedia capabilities.
The said server means can comprising Content servers and Computing Servers.
In accordance with an aspect the integrated convergence platform with local mouse implementation facility comprises any or all of the following depending on the users' application requirement -
General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units. The system can also be adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
In accordance with another aspect the integrated convergence platform with local mouse implementation facility comprising the following hardware blocks -
Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means; Serial Peripheral Interface; SPORT means; Peripheral Component Interconnect Bus means;
USB Controller; Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller; Speaker/MIC means; Interface to Monitor and / or TV;
A power source,
One Voice over Internet Protocol (VoIP) Port An optional PCI expansion slot, modem, Ethernet switch, and Infra-Red Receiver In accordance with yet another aspect of the invention there is provided a computer readable information recording medium which records a program /algorithm for providing an integrated convergence platform for voice, video, and data applications including computing, communication and entertainment applications with local mouse implementation facility as discussed above. According to yet further aspect of the present invention there is disclosed a remote windows desktop connectivity from embedded systems with constrained resources comprising:
a remote desktop protocol operatively connecting to windows terminal server at one end and to a wrapper means at the other end ;
said wrapper means disposed there between said remote desktop protocol and a custom display hardware device having embedded graphical operations.
In the above remote windows desktop connectivity for input events, the remote desktop protocol is adapted to directly communicate with said hardware and comprises a key map wrapper for identifying the key strokes and converting the same to necessary format.
In particular, the remote windows desktop connectivity comprises a remote desktop protocol network layer adapted for interacting with Windows Terminal Server, a remote desktop protocol decoder means adapted to decode the data that passes to the wrapper means, said wrapper means provided as an interface between said display device and said decoder means.
The remote desktop protocol display device comprises a graphical Operations Core operatively connected to a Display Driver and a Device Cache said Display Driver in turn operatively connected to an output means whereby the core interacts with the Display Driver to pass the output to the appropriate output device. The graphical operations core comprises all graphical operations including preferably memory blitting, text drawing. The embedded system for remote connectivity can have no operating system or adapted to run on embedded Linux operating system.
According to an aspect the remote windows desktop connectivity can comprise plurality of said remote desktop protocols and remote user access desktop device comprising embedded customized software for providing computing, entertainment and voice capabilities.
Preferably, a device management console for managing said plurality of user access terminals/devices is provided. Advantageously, the said user access terminals /devices are free of any moving components and the device operates menu based graphical user interface (GUI). The said device connects to said operating system server also providing computing applications as one of its features. Importantly, device comprises embedded software specifically developed to control and provide the said features. In particular, such devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration. Preferably, the devices are application class computer access terminals with multimedia capabilities.
The said windows terminal server means can comprising Content servers and Computing Servers.
The remote windows desktop connectivity can be included in systems comprising any or all of the following depending on the users' application requirement -
General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units.
The remote windows desktop connectivity including remote access protocol adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
The said remote windows desktop connectivity include remote access protocol with the following hardware blocks -
Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means; S erial P eripher al Interface;
SPORT means;
Peripheral Component Interconnect Bus means; USB Controller; Audio Coder-Decoder means; Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means; Interface to Monitor and / or TV; A power source,
One Voice Over Internet Protocol (VoIP) Port An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
In accordance with another aspect there is provided a computer readable information recording medium which records a program /algorithm for providing remote windows connectivity from an embedded platform using the said remote windows desktop connectivity as detailed above.
It is thus possible by way of the above system of local mouse implementation to impart better and effective user-friendly characteristics to thin client computing system. In particular, the above system of the invention involving the local mouse implementation facility would provide for a combination of VNC technology and graphic operation technology in thin client computing environment. Thus the disadvantages and limitation in effecting transfer of mouse update from the server and delay involved between the movement of the mouse by the user and the actual update on the screen can be avoided. Also, the system architecture of the invention would enable achieving mouse movement generated by the user almost instantaneously without requiring the user to lose control over the screen temporarily. The system would take care of constraints of thin client computing environments such as the integrated convergence platform for voice, video and data application and bring in the much required compatibility between the server and the local access /terminal and overcome the bottleneck of implementing the local mouse technology due to unsupported packet format of existing VNC technology.
Moreover, the constraints of access terminal / PC with limited resources are found to have problems to process data due to memory limitation. This has been taken care of at the server by stripping the unused portion of the mouse area and sending only the modified area for the mouse drawing operation. Also the server side changes are adapted such that it does not affect other regular X-86 client's connected to it and only the user terminal / PC with limited resources have the effect of such change in the server configuration. In accordance with the further aspect of the invention the windows desktop connectivity from an embedded system having constraint resources such as small processor and small memory have been achieved in an manner whereby it is possible to avoid the dependencies on X- server and make for the much desired effective use of embedded environment with constraint resources where it is taxing to run heavy applications like X-server. Using such system it would now be possible for the embedded devices to be able to support windows desktop connectivity, thus saving cost on additional resources such as processing capabilities, memory etc.
Advantageously, the above remote desktop protocol eliminates any known dependency on X- 11 and X-server. The wrapper between the present RDP and a custom display hardware has been provided for such purpose in the system. The graphical operations are directly embedded on disk play hardware. For input events too the present RDP communicates directly to hardware and has a key map wrapper for identifying the keystrokes and converting the same to the necessary format.
Thus by way of the above system the embedded devices would be able to support windows desktop connectivity thus saving cost on additional resources.
The details of the invention, its object and advantages are explained hereunder in greater detail in relation non-limiting exemplary illustration as per the following accompanying figures
BRIEF DESCRIPTION OF THE ACCOMPANYING FIGURES Figure 1 : Illustrates a multimedia capable server based computing environment provided by the convergence platform as in our co-pending PCT International application;
Figure 2: Illustrates a system block diagram describing the various high level system blocks as used in the system of our said co-pending application;
Figure 3: is a block diagram illustrating the mouse movement in a typical existing VNC protocol as per the existing system;
Figure 4: is a block diagram illustrating the mouse movement involving the VNC technology and graphic operation technology in accordance with the present invention for use with the embedded systems in particular the systems as per figures 1 and 2 above;
Figure 5: is an illustration of the architecture diagram of remote desktop application with X- server of the existing art;
Figure 6: is an illustration of architecture diagram of remote desktop application without X- server and involving the wrapper in accordance with the present invention; and
Figure 7: is an illustration of the display device used in the system involving the remote desktop application with wrapper and without X-server in accordance with the invention.
Reference is first invited to accompanying figure 1 which illustrates the provision of thin client computing environment in accordance with the system providing for the simple and effective operative connection to plurality of access terminal / devices and the triple play application area comprising content server [2], computing server [3] etc.
As clearly illustrated in Figure 1 the simplified convergence platform provides for the much desired server centric computing environment by operative communication with plurality of access terminal / devices (AT) [6] and with the required servers for the triple play applications including the content server (CS) [2] and computing server (CS) [3]. As further illustrated in same figure, the management of the plurality of the access terminal (AT) [6] is achieved by means of the device management console (DMC) [5]. Importantly, such device management console (DMC) [5] is simple and user friendly since the same is required to only manage very simplified access terminal (AT) [6] which do not use any moving parts / components or have any heat generating resources. As also illustrated in said figure, the operative connection of the convergence platform to the operative servers and the management console is attend through LAN / WAN environment.
Reference is now invited to accompanying figure 2 which illustrates the system block diagram describing the various high level system blocks. As apparent from said figure, the system basically comprises of General Purpose Processors (GPP) [7] to communicate over the network, handle tasks and manage the peripheral interfaces and a media processor (MP) [8] which is also a central processing unit adapted to handle media specific part of the platform.
The memory block (RAM) [9] indicates random access memory modules used by one or both the processors [8]. It is required for temporary calculation for running programs. A flash means (FM) [10] which is basically a persistence storage area for software code that is also used along with optional provision of a smart card reader (SMR) [11] for high security environments.
The various high level system blocks used in the convergence platform and its purposes are discussed hereunder in greater detail.
The General Purpose Processor (GPP) [7] is a general purpose CPU (Central Processing Unit). Its main function is to communicate over the network, handle tasks and manage the many peripheral interfaces available to it. It may or may not be present in all configurations of the hardware platform described, In which case the Media Processor [8] takes over the general purpose processing.
The Media Processor (MP) [8] is the CPU that handles the media specific part of the platform and interfaces' to output devices like TV, monitor [34], speaker [33], etc. For example, the access devices [6] in the present system, when used in a server centric environment, have full multimedia capability unlike thin clients, allowing the use of the device as a media access client or as Video conferencing equipment. Where the General Purpose Processor [7] is unavailable in certain configurations, it takes over its role as well.
The System Bus (SB) [13] is a set of wires that connects together various components with the CPU.
A Smart Cards Reader means (SCR) [11] is preferably provided to read authentication cards such as Credit Cards, ATM Cards, Identity ' Cards, Access Cards, etc. In high security environments, where typically a user needs to be identified with much more than a user name and password, this can be used to check if a user is actually producing a valid card. The Smart Card Reader (SCR) [11] is basically a hardware device used in the system that is used to read information from Smart Cards [H].
The Simplified Convergence Platform design offers the following hardware blocks —
The Ethernet MAC (EMAC) [14] or the EMAC [14] which is the Ethernet controller that is responsible for the hardware connectivity part of the Networking Subsystem.
Universal Asynchronous Transmitter and Receiver (UART) [16] that is used for serial communication.
The Serial Peripheral Interface (SPI) [18] is a serial bus used to communicate with other SPI [18] capable devices. Since the media processor [8] has this capability, it is used to communicate between the General Purpose [7] and Media processors [8].
Means identified as (SPORT) [19] comprising a Serial port. In the system, it is used for inter- processor communications.
The Peripheral components Interconnect Bus (PCI) are an industry standard bus that can be used to connect various commercially available expansion slots that add functionality to the system.
The component USB COntr [23] is the USB controller that helps communicate between the General Purpose Processor [7] and various USB devices. In the system, the USB controller [23] controls devices such as the Mouse [28] , the Keyboard [27] , the Fingerprint scanner [29] used for user authentication and Web Camera [26] used for Video Conferencing.
The (PCI) expansion slot [24] is an optional block that can be present to accommodate commercially available hardware expansion slots like graphics cards or sound cards.
The Audio CODEC [31] is an Audio Coder-Decoder that helps with digital audio encoding and decoding. Internally, the system, like all computers deals with digital signals. But any devices like speakers [33] and VGA monitors [34] need analog signals. The Video DAC [32] or Digital to Analog converter is one such device that converts the digital signals delivered by the Media processor [8] into analog signals that the VGA monitor can display.
The system further includes speakers/mic (SP) [33] which is a normal set of speakers and microphone used commonly in computers. Also monitors (MN) [34] comprise a regular PC monitor.
The system is powered by a power supply (PS) [35] which is the unit that supplies DC power to the whole system.
Reference is now invited to Figure 3 which illustrates the mouse movement and the cause of delayed response from the server in the existing VNC protocol as per accompanying Figures 1 and 2. As shown in the figure the mouse movement is sent to the server (SV) and mouse update is also a part of the screen update. Thus since the mouse update is from the server (SV) in such existing system there is delay involved between the movement of the mouse by the user and the actual update of the screen. As a result, mouse movement is not synchronized with respect to the hand and eyes of the user (CL1/CL2)). This is more in case of heavy flash or screen updates where the traffic in the network is heavy which lead to further slowing of the mouse response in such conventional system.
Reference is now invited to Figure 4 which illustrates the manner of achieving an integrated convergence platform with local mouse implementation facility in accordance with the present invention. As desired, the block at the client shows that the client (CLl) is sending the mouse packet to the server (SV) as well as using the data within itself for drawing the mouse. Only when there is a change in mouse shape, the server is adapted to send the new shape to the client as a separate packet.
The client (CLl) identifies the mouse packet and the screen update packet as to different packet type and handles it differently. If the packet is corresponding to the mouse packet, it does graphic operation and draws the mouse locally. Otherwise the screen update is drawn as in the original VNC protocol. The methodology of operation of thin client computing systems as per Figures 1 and 2 with possible local mouse implementation in accordance with the present invention is detailed further hereunder:
The initial mouse shape is received from the server and stored locally in the media processor. Whenever the user moves the mouse, the networks processor captures the coordinates. The processors send the coordinates to both the media processor and the server.
The media processor draws the mouse on the screen with respect to the X and Y coordinates send by the network processor. If there is a mouse shape change, a server recognizes the change and sends the pattern and the mask to the client. The client (network processor) receives this as a separate mouse packet and gives the data to a media processor. The media processor recognizes and handles the mouse packet and draws the packet on titie screen as if it is graphic operation.
The other screen update comes from the server as a regular VNC screen updates and is decoded and displayed by the client. Since the mouse movement is recognized and handled on priority basis, the mouse movements are faster thus the technology solves the problem in delay in mouse response due to network traffic. This achieves a PC like experience for the user.
At the server side, the X-Il cursor option sends the mask as 1 bit information. Whereas packet type use in the access terminal PC supports only 2 byte information. Hence that is modified in the server to support 2 byte mask information. Also, the access device / PC of the system which has constraints to process only 1,000 words of data for the mouse packet due to memory limitation. This is taken care at the server by stripping the unused portion of the mouse area and sending only the modified area for mouse drawing operation. Also the server side changes are done such that it does not affect the regular X-86 client connected to it but only the clients/ user terminal PC with constraint resources have the effect of this change.
Reference is now invited to accompanying figure 5 which shows a conventional Remote Desktop Protocol (RDP) which communicates with X-server (XSV) with the help of X-11 (XIl) library. X-Il serves as a wrapper between X-clients and X-server, isolating the underlying graphical implementation for application calls. X-server initiates input / output devices and waits for request from X-clients. The X-clients like remote desktop connects to the X-server (XSV) and starts sending graphical operations such as text, bitmaps, blitting etc. X-server is also responsible for receiving inputs from input devices like mouse, keyboard etc. on behalf of the X-clients.
The remote desktop protocol avoiding the dependency on X-11(XIl) and X-server (XSV) in accordance with the present invention is illustrated in accompanying figure 6. As apparent from said Figure 6 in the present system a wrapper (WR) between the remote desktop protocol and custom display hardware (CDH) has been provided. This enables the graphical operations directly embedded on the display hardware. For input events too, the remote desktop protocol involving the wrapper means of the invention directly communicates with hardware and has a key map wrapper for identifying the key strokes and converting the same to necessary format.
As clearly apparent from the Figure 6, in case of remote desktop protocol involving wrapper (WR) of the invention, the remote desktop protocol network layer (RDPNL) is responsible for interacting with windows terminal server (WTSV) . The remote desktop protocol decoder is adapted to decode the data that passes to wrapper, and interface between the architecture as clearly illustrated further in figure 6.
Reference is now invited to accompanying figure 7 which illustrate by way of block diagram the display device architecture used in the system of the invention. As shown in the figure 7 the graphical operation code (GOC) contains all the graphical operations including memory blitting, text drawing to name a few. This core (GOC) interacts with display driver (DD) to pass the output to the appropriate output device (OD). It is thus possible by way of the above connectivity to remove dependency on X-server and thereby advancing the utility of the same in the embedded environment with constraint resources where it is taxing to run heavy application like X-server. By way of the invention embedded devices would be able to support window desktop connectivity thus saving cost all additional resources and processing capabilities, memories etc. Importantly, the system can be advantageously used as architecture for thin clients with minimal processing capabilities.

Claims

WE CLAIM:
1. An integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications comprising:
plurality of user access terminal/devices having respective network processor with constrained resources with or without additional network processors of varied capabilities in a server centric Computing environment;
a LAN and/or WAN operating connection to server means comprising a network server and a media server ;
a local mouse implementation facility comprising said media processor adapted to store locally the initial mouse shape received form the server; said network processor adapted to capture the coordinates whenever the user moves the mouse and to forward the coordinates to the said media processor and the server; said media processor adapted to draw the mouse on the screen with respect to the x and y coordinates ;said server adapted to recognize any change in shape of the mouse and forward the pattern and mask to the to the network processor; the network processor further adapted to receive the same as a separate mouse packet and forward the data to the media processor; and said media processor adapted to recognize and handle the mouse packet and draw the packet on the screen as a graphics operation.
2. An integrated convergence platform with local mouse implementation facility as claimed in claim 1 wherein the network server is adapted to provide other screen updates as a regular VNC screen update and is decoded and displayed by the client/network processor.
3. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 or 2 wherein the network server is adapted to support 2 byte, mask information for communicating with said access terminal/devices having respective network processor with constrained resources.
4. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 3 wherein the network server is adapted to strip the unused portion of the mouse area and forward only the modified area for mouse drawing operations to said access terminal/devices having respective network processor with constrained resources.
5. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 4 wherein the network server is modified only to the extent of supporting the access terminal/devices having respective network processor with constrained resources.
6. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 5 wherein the client/network processor is adapted to forward mouse packet to the server and also use the data within itself for drawing the mouse.
7. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 6 wherein the client/network processor is adapted to identify the mouse packet and the screen updates as two different packet types and handles it differently such that when the packet is corresponding to the mouse packet it does graphic operation and draws the mouse locally, otherwise, the screen update packet is drawn as in the original VNC protocol.
8. An- integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 7 adapted for required server scalability for said access terminal/devices having respective network processor with constrained resources and also all other clients connected to the same server through additional network processors of varied capabilities.
9. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 8 comprising said platform having embedded customized software for providing computing, entertainment and voice capabilities.
10. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 9 comprising a device management console for managing said plurality of user access terminals/devices.
11. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 10 wherein said user access terminals /devices are free of any moving components.
12. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 11 wherein said device operates menu based graphical user interface (GUI).
13. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 12 wherein the said device connects to an operating system server also providing computing applications as one of its features.
14. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 13 wherein said device comprises embedded software specifically developed to control and provide the said features.
15. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 14 wherein said devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
16. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 15 wherein said devices are application class computer access terminals with multimedia capabilities.
17. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 16 wherein said server means comprising Content servers and Computing Servers.
18. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 17 comprising any or all of the following depending on the users' application requirement -
General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units.
19. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 18 adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
20. An integrated convergence platform with local mouse implementation facility as claimed in anyone of claims 1 to 19 comprising anyone or more of the following hardware blocks - Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means;
Serial Peripheral Interface;
SPORT means;
Peripheral Component Interconnect Bus means; USB Controller;
Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller;
Speaker/MIC means;
Interface to Monitor and / or TV; A power source,
One Voice Over Internet Protocol (VoIP) Port
An optional PCI expansion slot, modem, Ethernet switch, and
Infra-Red Receiver
21. A computer readable information recording medium which records a program /algorithm for providing an integrated convergence platform for voice, video, and data applications including computing, communication and entertainment applications with local mouse implementation facility as claimed in claim 1.
22. A remote windows desktop connectivity from embedded systems with constrained resources comprising:
a remote desktop protocol operatively connecting to windows terminal server at one end and to a wrapper means at the other end ;
said wrapper means disposed there between said remote desktop protocol and a custom display hardware device having embedded graphical operations.
23. A remote windows desktop connectivity as claimed in claim 22 wherein for input events the remote desktop protocol is adapted to directly communicate with said hardware and comprises a key map wrapper for identifying the key strokes and converting the same to necessary format.
24. A remote windows desktop connectivity as claimed in anyone of claims 22 to 23 comprising a remote desktop protocol network layer adapted for interacting with Windows
Terminal Server, a remote desktop protocol decoder means adapted to decode the data that passes to the wrapper means, said wrapper means provided as an interface between said display device and said decoder means.
25. A remote windows desktop connectivity as claimed in anyone of claims 22 to 24 wherein said remote desktop protocol display device comprises a graphical Operations Core operatively connected to a Display Driver and a Device Cache said Display Driver in turn operatively connected to an output means whereby the core interacts with the Display Driver to pass the output to the appropriate output device.
26. A remote windows desktop connectivity as claimed in anyone of claims 22 to 25 wherein said graphical operations core comprises all graphical operations including preferably memory blitting, text drawing.
27. A remote windows desktop connectivity as claimed in anyone of claims 22 to 26 wherein the embedded system is selectively having no operating system or adapted to run on embedded Linux operating system.
28. A remote windows desktop connectivity as claimed in anyone of claims 22 to 27 comprising plurality of said remote desktop protocols and remote user access desktop device comprising embedded customized software for providing computing, entertainment and voice capabilities.
29. A remote windows desktop connectivity as claimed in anyone of claims 22 to 28 comprising a device management console for managing said plurality of user access terminals/devices.
30. A remote windows desktop connectivity as claimed in anyone of claims 22 to 29 wherein said user access terminals /devices are free of any moving components.
31. A remote windows desktop connectivity as claimed in anyone of claims 22 to 30 wherein said device operates menu based graphical user interface (GUI).
32. A remote windows desktop connectivity as claimed in anyone of claims 22 to 31 wherein the said device connects to said operating system server also providing computing applications as one of its features.
33. A remote windows desktop connectivity as claimed in anyone of claims 22 to 32 wherein said device comprises embedded software specifically developed to control and provide the said features.
34. A remote windows desktop connectivity as claimed in anyone of claims 22 to 33 wherein said devices are equipped with specially developed management software for changing configuration parameters and/or updating the device configuration.
35. A remote windows desktop connectivity as claimed in anyone of claims 22 to 34 wherein said devices are application class computer access terminals with multimedia capabilities.
36. A remote windows desktop connectivity as claimed in anyone of claims 22 to 35 wherein said windows terminal server means comprising Content servers and Computing Servers.
37. A remote windows desktop connectivity as claimed in anyone of claims 22 to 36 comprising any or all of the following depending on the users' application requirement - General Purpose Processor having random access memory means; media processor having random access memory means; flash storage means for storing software codes; and system bus for operative connection of various components with the central processing units.
38. A remote windows desktop connectivity as claimed in anyone of claims 22 to 37 including remote access protocol adapted for embedded software and / or hardware as an interface to smart card reader and /or biometric sensor means to secure access to the computing /content as provided by the platform.
39. A remote windows desktop connectivity as claimed in anyone of claims 22 to 38 comprising remote access protocol with the following hardware blocks -
Ethernet MAC connectivity;
Universal Asynchronous Transmitter and receiver means; Serial Peripheral Interface; SPORT means; Peripheral Component Interconnect Bus means;
USB Controller; Audio Coder-Decoder means;
Video Digital to analog converter and / or VGA Controller; Speaker/MIC means; Interface to Monitor and / or TV;
A power source,
One Voice Over Internet Protocol (VoIP) Port An optional PCI expansion slot, modem, Ethernet switch, and Infra-Red Receiver
39. A computer readable information recording medium which records a program /algorithm for providing remote windows connectivity from an embedded platform using the said remote windows desktop connectivity as claimed in claim 22.
40. An integrated convergence platform with local mouse implementation facility adapted for voice, video, and data applications including computing, communication and entertainment applications and a remote windows desktop connectivity from embedded systems with constrained resources substantially as herein described and illustrated with reference to the accompanying figures.
PCT/IN2006/000163 2006-05-12 2006-05-12 Tracking mouse locally and microsoft windows connectivity algorithm WO2007132471A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/IN2006/000163 WO2007132471A1 (en) 2006-05-12 2006-05-12 Tracking mouse locally and microsoft windows connectivity algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IN2006/000163 WO2007132471A1 (en) 2006-05-12 2006-05-12 Tracking mouse locally and microsoft windows connectivity algorithm

Publications (1)

Publication Number Publication Date
WO2007132471A1 true WO2007132471A1 (en) 2007-11-22

Family

ID=38693596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IN2006/000163 WO2007132471A1 (en) 2006-05-12 2006-05-12 Tracking mouse locally and microsoft windows connectivity algorithm

Country Status (1)

Country Link
WO (1) WO2007132471A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8185581B2 (en) 2009-05-19 2012-05-22 Nholdings Sa Providing a local device with computing services from a remote host
US8756293B2 (en) 2007-04-23 2014-06-17 Nholdings Sa Providing a user with virtual computing services

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5953005A (en) * 1996-06-28 1999-09-14 Sun Microsystems, Inc. System and method for on-line multimedia access

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5953005A (en) * 1996-06-28 1999-09-14 Sun Microsystems, Inc. System and method for on-line multimedia access

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8756293B2 (en) 2007-04-23 2014-06-17 Nholdings Sa Providing a user with virtual computing services
US9277000B2 (en) 2007-04-23 2016-03-01 Nholdings Sa Providing a user with virtual computing services
US8185581B2 (en) 2009-05-19 2012-05-22 Nholdings Sa Providing a local device with computing services from a remote host
US8478817B2 (en) 2009-05-19 2013-07-02 Nholdings Sa Providing a local device with computing services from a remote host

Similar Documents

Publication Publication Date Title
CN106713485B (en) Cloud computing mobile terminal
US8762544B2 (en) Selectively communicating data of a peripheral device to plural sending computers
EP3046331B1 (en) Media control method and system based on cloud desktop
CN102946409A (en) Method, system of sending single terminal user experience from a plurality of servers to clients
CN102413150A (en) Server and virtual desktop control method and virtual desktop control system
TW201322000A (en) Device linking
CN110784525A (en) Cloud mobile phone control method, system and storage medium based on H5 webpage technology
CN102033726A (en) Display method and server
EP2308023B1 (en) Virtual media device
CN108055304B (en) Remote data synchronization method, device, server, equipment and storage medium
US20140108940A1 (en) Method and system of remote communication over a network
CN112221127B (en) Multi-point control method and device for cloud game
US9569161B2 (en) Remote visualization and control for virtual mobile infrastructure
WO2011120329A1 (en) Computer and method to realize the coupling between computers
CN113489805A (en) Docking method, device, equipment and storage medium of cloud desktop system
US20120166585A1 (en) Apparatus and method for accelerating virtual desktop
US20160112528A1 (en) Method and System for Serving a Virtual Desktop to a Client
WO2020093843A1 (en) Distributed multi-terminal and multi-network supporting system for android online game
CN113038192B (en) Video processing method and device, electronic equipment and storage medium
US6941385B2 (en) Server transferring only image display data for a part of a display image which is to be updated for associated terminals
TWI592874B (en) Network server system
CN103209209B (en) Based on applying virtual implementation method and the system of the transmission of Windows System and Network
US8984540B2 (en) Multi-user computer system
WO2007132471A1 (en) Tracking mouse locally and microsoft windows connectivity algorithm
CN111143017B (en) Cloud operation system interaction processing method, client and cloud operation system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 06756261

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 1156/CHENP/2008

Country of ref document: IN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06756261

Country of ref document: EP

Kind code of ref document: A1