Date: Mon, 01 Feb 2016 09:10:41 -0700 From: Don whY <Don.whY@gmx.com> To: freebsd-hackers@freebsd.org Subject: Re: IoT OS Message-ID: <56AF8381.9040307@gmx.com> In-Reply-To: <CAP8Xrcst%2Bq8VZ8e2A2Rb0w_C_2OX83x%2BdONPxTpsFBQF7JQvtw@mail.gmail.com> References: <CAP8XrcszH-KroGv3nJo5AyH=Mh%2BAe6YhAKNi4SuEEs9ss9M8Rg@mail.gmail.com> <56A10892.2090308@rlwinm.de> <F37089F2-6467-4366-8AE9-C8BD0BF6E6F7@gmail.com> <CAP8Xrcst%2Bq8VZ8e2A2Rb0w_C_2OX83x%2BdONPxTpsFBQF7JQvtw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[stuff elided, recipients trimmed] On 1/21/2016 11:04 AM, Mathieu Prevot wrote: >>>> On 21/01/16 17:19, Mathieu Prevot wrote: >>>> I would like to connect several connected object (with homogeneous or >>>> heterogenous hardare: intel edison, samsung artik, apple AX, intel core, >>>> etc) so the calculation needs, the storage/memory, the connection, etc are >>>> decoupled; hence we can reach an ecosystem with several clouds. Most of these are reasonably heavyweight nodes. I.e., you aren't talking about lightweight sensor motes. >>>> How do you recommend to reach that ? from the kernel, a module, or >>>> eventually a software ? > Say all objects are connected peer to peer with wifi, some of them are > connected to internet through gsm network or wifi to a box. These object > are moving in space, and for some reasons, connections are dynamical and > can be severely impaired or lost. So, you are expecting NONE of them to be precious nodes? What guarantees do you propose regarding interconnectivity? Will there *always* be a path through the mesh to The Internet? Or, to some_precious_node? How do you expect to handle the absence of that path? Or, the mesh partitioning into two or more disjoint meshes? > They have incoming local streams of data (eg HD videos, accelerometer, GPS, > other wifi and gsm signals, etc). > > I would like to abstract the CPU layer, storage layer, and internet > connection so that in realtime results of one of my objects are saved if > this object dies, so that if one of the object giving internet access to > the group loose its connection, the redundancy allows the group of object > not to lose internet connection. Nothing is going to magically do this *for* you. If you want to have reliability through redundancy, you need to provide that redundancy, somewhere and by some means. You also casually slip the term "realtime" into the description. What are your timeliness constraints on the data and its handling? Especially in conditions of "fault"? Do you expect the "system" to enforce these constraints? Or, do you just assume "realtime" means "current"? > Can I consider these as different load balancing layers ? Do you recommend > to implement this at the kernel layer or at an API layer ? Can I see that > as a lightweight cluster ? I've been proceeding along the lines of a homogenous set of hardware (simply because it means I have less hardware to *design*/build) of reasonably heavyweight capabilities (i.e., no motes). I've built an RTOS that lets me dynamically move "tasks" between nodes at run time. In my world, everything is wired (cuz I don't want to have to deal with the potential for trivial DoS attacks based on remote RF jamming) and PoE powered. This lets me hide devices in places where a "wall wart" wouldn't be convenient. It also lets me power nodes up/down as needed. E.g., if I need more computational horsepower, I can bring a node on-line solely for it's "CPU" -- even though it's particular I/O's may not be needed at this time. (and, because I can move a "task" at will, I can then push arbitrary "tasks" off to that node in the "balancing act") I've split the design into three distinct layers: - RTOS (to provide RT guarantees, mechanisms, etc.) - core services (things that "applications" would tend to require) - applications (intended to be greatly abstracted) A homogenous environment makes the design of the RTOS simpler. I don't have to deal with endian-ness issues, different data representations, on-the-fly marshalling conversions in the RPC mechanism, etc. It lets the *distributed* system more closely resemble an SMP implementation. Running on bare metal, the RTOS can eek out a bit more performance (to compensate for some of the inefficiency of a distributed system). The core services run atop the RTOS and can benefit from its rich API. E.g., the details of implementing redundant tasks can be hidden from the application layer so it just looks like a "system call" -- the application need not worry about *where* the redundant task gets spawned... nor how the data streams get routed/coordinated. The application layer runs in VM's on each node. So, I can just "move" the contents of a particular "application" to a VM running on another node and resume execution, there (the system deals with ensuring all the communication paths move *with* the application). Timeliness guarantees get harder to satisfy as you move up from the metal. E.g., communication delays vary for local IPC vs. RPC -- and what was an IPC a moment ago may become an RPC if one endpoint happens to be relocated as part of the load shifting. As a result, most of the true "real-time" issues are handled in the core services -- as they are designed to be aware of these run-time issues in a way that "application developers" are (deliberately) not. My system/hardware design implicitly assumes some nodes will always be accessible (e.g., they are all connected/powered from the network switch so the switch, at the very least, is present in all configurations!). This allows other "key services" to be guaranteed. E.g., I have a single persistent store that satisfies all of the needs of the nodes in the system (with redundancy implemented *within*). Likewise, precise clock synchronization so the movement of "tasks" doesn't introduce relative timing anomalies. If you can't make any such guarantees, then you'll have to ensure each node is capable enough to provide any guarantees that *it* requires along with those of the other nodes in whichever mesh (or portion thereof) that it resides. > I think the API is more flexible, especially if I have an heterogeneous (by > CPU, OS) set of connected object. However, working at the kernel level > allows existing programs not to be rewritten. What are your thoughts ? What "existing programs"? Are you envisioning desktop applications migrating into your mesh? Or, the data sourced from the mesh integrating with desktop apps? Or, desktop apps sourcing data that the mesh *sinks*? > Do you recommend another list ? Probably -- though I'm not sure which one... Good luck!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56AF8381.9040307>