Date: Fri, 22 Jan 2016 17:08:32 +0000 From: "Rang, Anton" <anton.rang@isilon.com> To: Mathieu Prevot <mathieu.prevot@gmail.com>, NGie Cooper <yaneurabeya@gmail.com> Cc: Jan Bramkamp <crest@rlwinm.de>, "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>, "freebsd-current@freebsd.org" <freebsd-current@freebsd.org> Subject: RE: IoT OS Message-ID: <F21EDC44C64DB34B90AF485AC3CEDD4B82F7913D@MX104CL01.corp.emc.com> In-Reply-To: <CAP8Xrcst%2Bq8VZ8e2A2Rb0w_C_2OX83x%2BdONPxTpsFBQF7JQvtw@mail.gmail.com> References: <CAP8XrcszH-KroGv3nJo5AyH=Mh%2BAe6YhAKNi4SuEEs9ss9M8Rg@mail.gmail.com> <56A10892.2090308@rlwinm.de> <F37089F2-6467-4366-8AE9-C8BD0BF6E6F7@gmail.com> <CAP8Xrcst%2Bq8VZ8e2A2Rb0w_C_2OX83x%2BdONPxTpsFBQF7JQvtw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
>Say all objects are connected peer to peer with wifi, some of them are con= nected to internet through gsm network or wifi to a box. >These object are moving in space, and for some reasons, connections are dy= namical and can be severely impaired or lost. > >They have incoming local streams of data (eg HD videos, accelerometer, GPS= , other wifi and gsm signals, etc). > >I would like to abstract the CPU layer, storage layer, and internet connec= tion so that in realtime results of one of my objects are saved >if this object dies, so that if one of the object giving internet access t= o the group loose its connection, the redundancy allows the group of object= not to lose internet connection. > >Can I consider these as different load balancing layers ? Do you recommend= to implement this at the kernel layer or at an API layer ? >Can I see that as a lightweight cluster ? > >I think the API is more flexible, especially if I have an heterogeneous (b= y CPU, OS) set of connected object. However, working at the kernel level al= lows existing programs not to be rewritten. >What are your thoughts ? =3D=3D=3D OK, I think I understand your question now. This isn't the right list for it, though I'm not sure where the right place= to go would be -- it's not FreeBSD-specific, in any case. There are academ= ic research groups looking into this type of problem; for instance, in the = area of sensor networks (ACM Transactions on Sensor Networks covers some of= these areas). There may be USENET groups which cover this area. To cover your three areas, which I think require somewhat different solutio= ns -- (a) CPU layer. I don't really recommend trying to abstract this. You coul= d use a virtual machine to hide the underlying architecture, and checkpoint= state periodically, but this is likely to slow down execution too much to = be useful. If the issue that a service may become unavailable, I'd recomme= nd a middleware layer which can detect this and recover by starting a new i= nstance of the service. Middleware layers like ZeroMQ, and clustering softw= are, may be a useful starting point. This does mean that stateful connecti= ons (like reading a video stream) won't recover cleanly, though; the client= would need to reconnect to attach to the new instance of the service. If = you really need that, it's going to be hard. (b) Storage layer. Look into highly-available clustered storage solutions.= If you can use key-value or some other simplified storage model, do it. = There are clustered file systems but probably none freely available that wo= uld work on the scale you envision and give decent performance. There are = more alternatives if you're flexible about the format in which you're stori= ng data (e.g. replicated object stores). (c) Networking layer; or internet. If you can drop & re-establish a connect= ion, or if every node has its own IP address (IPv6), this should be pretty = straightforward; software could detect loss of connection and change the ro= uting used to go through a different system. If not, you'll be a bit limite= d since mirroring TCP state between nodes would be too slow. This is a case= where the existing operating system kernels are likely to do most of what = you need; you simply need to add a layer to detect routing problems and sel= ect a new internet gateway appropriately. I'd avoid implementing any clustering within the kernel, in part because if= you have a wide variety of objects you may not want the same kernel on all= of them, and in part because debugging & recovery is much harder. You're u= nlikely to want to run most existing software on such a system anyway (espe= cially if they have relatively weak processors); you're better off writing = to a set of clustering APIs for storage and state, at least. For networking= , as mentioned, you can likely use the existing TCP stack & just add contro= ls to redirect traffic as needed. -- Anton
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F21EDC44C64DB34B90AF485AC3CEDD4B82F7913D>