Leveraging innovation and technology to provide the highest level of security against emerging threats, MARSS’ intuitive solutions employ integrated sensor surveillance ... NiDAR Command, NiDAR 4D, NiDAR CUAS, NiDAR CUAS COMPACT, NiDAR X, NiDAR X ...

MARSS demos NiDAR-powered C2 systems to UK MoD

Amazon Great Indian Festival sale is ending today at midnight. Here are the best deals you can grab before the grand Amazon sale ends.

Amazon Great Indian Festival: Best deals to grab before sale ends at midnight

Berlin: INTERGEO EXPO and CONFERENCE has come to a successful close after three days in Berlin. The leading international trade fair for geodesy, geoinformation and land management presented itself as a magnet for the international geospatial community and for the diverse application areas of geodata-based solutions.

INTERGEO confirms its position as the leading platform for the geospatial industry

According to an industry analysis conducted by Fact.MR, in 2022, the worldwide LiDAR sensor market is projected to exceed a value of US$ 1,908.1 million, with a prospective growth rate of 12.7%. By the close of 2032,

LiDAR Sensor Market Demand is anticipated to Reach US$ 6,328.7 million by 2032 - Fact.MR

Back in July, the Enstone and Viry-based outfit made the case that because it was 30 horsepower inferior to its F1 rivals, it should be allowed to ramp up the performance.

F1 rejected Alpine's bid for engine power boost

LLNL's high-purity germanium gamma-ray sensor is part of a suite of instruments traveling to the largest asteroid in the solar system.

LLNL Instrument Making 2-Billion-Mile Journey To Asteroid

Players may opt TO play the PERM, this play TYPE IS simply a straight play of ALL the possible arrangements (permutations) of the number. FOR more info, visit the PCSO 4D Lotto page.

4D Lotto

The double-sized notch and the short-lived Soli sensor stand out as some of Google’s more out-there experiments. But despite the company’s path towards more refined and purposeful hardware ...

The Pixel 8 Pro's temperature sensor is too inaccurate, too inconvenient

Radars are already cemented into the automotive market as they provide popular ADAS features such as adaptive cruise control (ACC) and

Short-Range Radars Have Huge Potential in the Automotive Radar Market, Finds IDTechEx Research

As a photographer, I wanted to love the Pixel 8 Pro's new high-end camera technology. But some unpleasant photo surprises during my

The Pixel 8 Pro Has Camera Glitches. Here's How to Sidestep Them

4D Sensor

4dsensor

One Would Think The World Was Built Just as Much for Women as It Is For Men

Човек би си помислил, че светът е създаден точно толкова за жените, колкото и за мъжете

Es podria pensar que el món s'ha construït tant per a les dones com per als homes

Uno podría pensar que el mundo fue construido tanto para mujeres como para hombres

世界は男性のためだけでなく女性のためにも作られていると思う人もいるだろう

შეიძლება ვიფიქროთ, რომ სამყარო ისევე აშენდა ქალებისთვის, როგორც მამაკაცებისთვის

S-ar crede că lumea a fost construită la fel de mult pentru femei ca și pentru bărbați

Mumwe Angafunga Kuti Nyika Yakavakwa Zvakangofanana nevakadzi sezvazvakaita kuvarume

Qofku waxa uu u malaynayaa in dunidu u dhisnayd sida dumarka ay ugu dhisan yihiin ragga

இந்த உலகம் ஆண்களுக்காகப் படைக்கப்பட்டதைப் போலவே பெண்களுக்காகவும் படைக்கப்பட்டுள்ளது என்று ஒருவர் நினைப்பார்.

Iisipin ng isang tao na ang mundo ay ginawa para sa mga babae gaya ng para sa mga lalaki

کوئی سوچے گا کہ دنیا عورتوں کے لیے اتنی ہی بنائی گئی ہے جتنی مردوں کے لیے ہے۔

Người ta nghĩ rằng thế giới được xây dựng cho phụ nữ nhiều như cho nam giới

What’s Next for Jupiter’s Fiery Moon, Io?

Turn Your Dumb Solar Inverter Into a Smart One With This Home Assistant Hack

Evaluating Panopticus: Performance, Robustness, and Component Analysis

Table of links ABSTRACT 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 7 IMPLEMENTATION We implemented Panopticus using Python and CUDA for GPU-based acceleration. All neural networks were developed using PyTorch [41] and trained on the training set in the nuScenes dataset [2]. Note that Panopticus is compatible with other 3D perception datasets such as Waymo [46], having sensor configurations similar to nuScenes. The neural networks for 3D object detection are developed based on Figure 9: Setup for mobile testbed and dataset. MMDetection3D [35]. For the camera motion network, we customized and trained [53] to produce consistent relative poses between two consecutive frames for any given camera view. For the object tracker, we modified SimpleTrack [36] to use velocity for 3D Kalman’s state transition model. We used the GPU-accelerated XGBoost library [12] and linear model in scikit-learn [43] for performance predictors, and PuLP [17] for the ILP solver. In the model adaptation stage, memory usage is profiled via tegrastats We optimized the performance of our multi-branch model in various ways. We used TensorRT [11] to modularize the neural networks and to accelerate the inference. Networks converted to TensorRT are optimized using floating-point 16 (FP16) quantization and layer fusion, etc. Camera view images assigned to the same networks, such as backbone network or DepthNet, are batch-processed together. Additionally, we used CUDA Multi-Stream [5] to process these multiple networks in parallel on an edge GPU. Meanwhile, we noticed an accuracy loss due to the simplistic modularization of our multi-branch model. Specifically, DepthNets and BEV head trained with a specific backbone network are incompatible with others. One-size-fits-all DepthNet is not practical since each backbone generates a 2D feature map of different sizes. Thus, our model includes DepthNet variants tailored to each backbone, ensuring accurate depth estimation. In contrast, the BEV head takes inputs of a consistent shape, regardless of the preceding networks—backbones and DepthNets. This allows us to train a universal BEV head compatible with any combination of preceding networks. We trained the BEV head from scratch, while the pre-trained backbones and DepthNets were fine-tuned. Authors:

Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. Table of links Table of links ABSTRACT ABSTRACT ABSTRACT 1 INTRODUCTION 1 INTRODUCTION 1 INTRODUCTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 2 BACKGROUND: OMNIDIRECTIONAL 3D OBJECT DETECTION 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3 PRELIMINARY EXPERIMENT 3.1 Experiment Setup 3.2 Observations 3.3 Summary and Challenges 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 4 OVERVIEW OF PANOPTICUS 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5 MULTI-BRANCH OMNIDIRECTIONAL 3D OBJECT DETECTION 5.1 Model Design 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6 SPATIAL-ADAPTIVE EXECUTION 6.1 Performance Prediction 5.2 Model Adaptation 6.2 Execution Scheduling 7 IMPLEMENTATION 7 IMPLEMENTATION 7 IMPLEMENTATION 8 EVALUATION 8 EVALUATION 8 EVALUATION 8.1 Testbed and Dataset 8.2 Experiment Setup 8.3 Performance 8.4 Robustness 8.5 Component Analysis 8.6 Overhead 9 RELATED WORK 9 RELATED WORK 9 RELATED WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 10 DISCUSSION AND FUTURE WORK 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 11 CONCLUSION AND REFERENCES 7 IMPLEMENTATION 7 IMPLEMENTATION We implemented Panopticus using Python and CUDA for GPU-based acceleration. All neural networks were developed using PyTorch [41] and trained on the training set in the nuScenes dataset [2]. Note that Panopticus is compatible with other 3D perception datasets such as Waymo [46], having sensor configurations similar to nuScenes. The neural networks for 3D object detection are developed based on Figure 9: Setup for mobile testbed and dataset. MMDetection3D [35]. For the camera motion network, we customized and trained [53] to produce consistent relative poses between two consecutive frames for any given camera view. For the object tracker, we modified SimpleTrack [36] to use velocity for 3D Kalman’s state transition model. We used the GPU-accelerated XGBoost library [12] and linear model in scikit-learn [43] for performance predictors, and PuLP [17] for the ILP solver. In the model adaptation stage, memory usage is profiled via tegrastats We optimized the performance of our multi-branch model in various ways. We used TensorRT [11] to modularize the neural networks and to accelerate the inference. Networks converted to TensorRT are optimized using floating-point 16 (FP16) quantization and layer fusion, etc. Camera view images assigned to the same networks, such as backbone network or DepthNet, are batch-processed together. Additionally, we used CUDA Multi-Stream [5] to process these multiple networks in parallel on an edge GPU. Meanwhile, we noticed an accuracy loss due to the simplistic modularization of our multi-branch model. Specifically, DepthNets and BEV head trained with a specific backbone network are incompatible with others. One-size-fits-all DepthNet is not practical since each backbone generates a 2D feature map of different sizes. Thus, our model includes DepthNet variants tailored to each backbone, ensuring accurate depth estimation. In contrast, the BEV head takes inputs of a consistent shape, regardless of the preceding networks—backbones and DepthNets. This allows us to train a universal BEV head compatible with any combination of preceding networks. We trained the BEV head from scratch, while the pre-trained backbones and DepthNets were fine-tuned. Authors: Jeho Lee
Chanyoung Jung
Jiwon Kim
Hojung Cha Authors: Jeho Lee Chanyoung Jung Jiwon Kim Hojung Cha Jeho Lee Jeho Lee Chanyoung Jung Chanyoung Jung Jiwon Kim Jiwon Kim Hojung Cha Hojung Cha This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. This paper is available on arxiv under CC by 4.0 Deed (Attribution 4.0 International) license. available on arxiv available on arxiv

How Panopticus Uses AI to Detect Objects in 3D

Scientist Finds Equatorial Ocean Turbulence Is Surprisingly Ordinary

no trees, just rocks Redux has become one of the most popular Flux implementations for managing data flow in React apps. Reading about Redux, though, often causes a sensory overload where you can’t see the forest for the trees. Presented below is a straightforward and opinionated workflow for implementing real life apps with Redux. It is shown by example with a step by step implementation walkthrough of an actual app. It attempts to apply the principles behind Redux in a practical way and detail the thought process behind every decision. An opinionated approach for idiomatic Redux has become more than just a library, it’s an entire . One of the reasons behind its popularity is its ability to accommodate different writing styles and many different flavors. If I’m looking for asynchronous actions should I use ? or maybe ? or ? Redux ecosystem thunks promises sagas There isn’t one right answer to which flavor is “best”. And there isn’t one right way to use Redux. Having said that, too much choice is . I want to present an flavor that I personally like. It’s robust, can deal with complicated real life scenarios — and most of all — it’s . overwhelming opinionated simple So let’s build an app! We need a real life example to walk through. As long as we’re being opinionated, the most interesting place on the Internet is . Let’s make an app that shows the most interesting posts there. Reddit On the first screen, we’ll ask the user for 3 topics they’re interested in. We’ll pull the list of topics from Reddit’s list of default front page . subreddits After the user makes a choice, we’ll show the list of posts from each of these 3 topics in a filterable list — all topics or just one of the 3. When the user clicks on a post in the list, we’ll show its contents. Setup Since we use React for the web (we might add React Native in a future post), our starting point will be , the official starter kit. We’ll also npm install , and . The result should be something like . Create React App redux react-redux redux-thunk this To get the boilerplate out of the way, let’s also quickly initialize the Redux store and hook up thunk middleware in : index.js The Flux circle of life in a Redux app One of the main things often missing from Redux tutorials is the grand picture and where Redux fits in. Redux is an implementation of the architecture — a pattern for passing data around in a React app. Flux Under classic Flux, app state is held within . Dispatched cause this state to change, afterwhich the that listen to these state changes will re-render themselves accordingly: stores actions views Flux simplifies life by making data flow in a direction. This reduces the as the codebase grows and becomes more complex. single spaghetti effect One of the difficulties with understanding Redux is the plethora of unintuitive terms like , and . It’s easier to see where they all fit in by placing them on the Flux diagram. These are simply the technical names of the various Redux constructs that implement the different parts of the cycle: reducers selectors thunks As you’ve probably noticed, other terms from the Redux ecosystem like and , are absent. This is intentional as they won’t play a significant role in our workflow. middlewares sagas Project directory structure We will organize our code according to the following top-level directory structure under : /src “Dumb” React components that have no knowledge of Redux /src/components “Smart” React components that are to our Redux store /src/containers connected Abstraction facades for external API (like backend servers) /src/services All Redux-specific code goes here, including all of our app /src/store business-logic The directory is organized by , each containing: store domain Reducer as a default export with all selectors as named exports /src/store/{domain}/reducer.js All the domain action handlers (thunks and plain object creators) /src/store/{domain}/actions.js A state-first approach Our app has two screens, we’re going to start with the first and let the user choose exactly 3 topics. We can begin implementing any point of the Flux cycle, but I’ve discovered it’s usually easiest for me to start with the . State So what does our screen need? app state topics We’ll need to hold the list of topics retrieved from the server. We’ll also need to hold the ID’s of the topics chosen by the user so far (max 3). It will be nice to hold them in the order selected, so in case we already have 3 and another is chosen, we can simply drop the oldest one. How will we structure this app state? There’s a list of actionable tips in my previous post — “ ”. Following the tips, this would be an appropriate structure: Avoiding Accidental Complexity When Structuring Your App State The topic URL will serve as a unique ID. Where will we hold this state? In Redux, the is the construct that holds state and updates it. We will organize our code by , so the natural place for this reducer will be: reducer domains /src/store/topics/reducer.js There’s some boilerplate to create a reducer, you can see it . Notice that in order to enforce of our state (as required by Redux), I’ve chosen to use an immutability library called . here immutability seamless-immutable Our first scenario After modeling state, I like to take a user scenario and implement it from start to finish. In our case let’s create our screen and display some topics as soon as it shows up. This component will be connected to our reducer, meaning it’s a “smart” component that’s aware of Redux. We’ll place it in topics /src/containers/TopicsScreen.js There’s some boilerplate to create a connected component, you can see it . Let’s also display it as the content of our . Now, when everything is set up, we can fetch some topics. here App component Rule: Smart components are not allowed to have any logic except dispatching actions. The scenario starts on the view’s . Since can’t run logic directly on the view, we will dispatch an action that will fetch the topics. This action is asynchronous of course, so it will be a : componentDidMount thunk In order to abstract the Reddit server API, we’ll create a new that does the actual network fetch. Its methods will be asynchronous so we can for the response. In general, I love the so much that my code hasn’t seen a direct use of in a long time. service await async await API promises The returns an , but our state structure stores the topics in a . The action body is a good place to do the conversion. In order to actually store the data in the state, we must invoke our reducer by dispatching a standard plain object action — . service array map TOPICS_FETCHED The full source for this stage is available . here A few words about services Services are used to abstract external API — in many cases server API like the one by Reddit. The benefit of this abstraction layer is that API’s change and we want to decouple our code as much as possible from them. If in the future Reddit decides to rename endpoints or change field names, we can hopefully contain the impact on our app to the service alone. provided Rule: Services must be completely stateless. This is a tricky rule in our methodology. Imagine what would happen if our Reddit API required login. We might be tempted to hold this login state in the by instantiating it with the login details. service This isn’t allowed in our methodology because all app state must be contained in the . Holding state in a would be a state leak. The acceptable approach in this case would be to provide every service function with login information as argument and hold the login state in one of our . store service reducers Implementing the service is fairly straightforward, you can see it . here Completing the scenario — reducer and view The plain object action arrives at our reducer and contains the freshly fetched as parameter. Our reducer doesn’t need to do much except save this data on the state: TOPICS_FETCHED topicsByUrl Notice the usage of to make this immutable change explicit and straightforward. Immutability libraries are of course optional, I prefer their syntactic sugar to . seamless-immutable object spread tricks After the state updates, our view needs to re-render. This means the view needs to listen on the part of the state it cares about. This is done with : mapStateToProps I decided that our view will render the list of topics using a separate component that takes a map and a (inspired by ). I’m using to prepare these two props in (they will later be passed on directly to the ). The two props can be derived from our state. Notice something interesting, I don’t access the state directly.. ListView rowsById rowsIdArray React Native mapStateToProps TopicsScreen ListView Rule: Smart components should always access state through selectors. are one of the most important constructs in Redux that people tend to overlook. A selector is a pure function that takes the global state as argument and returns some transformation over it. Selectors are tightly coupled to and are located inside . They allow us to perform a few calculations on data before it’s being consumed by the view. In our methodology, we take this idea even further. Every time anyone needs to access part of the state (like in ), they need to go through a selector. Selectors reducers reducer.js mapStatetoProps Why? The idea is to encapsulate the internal structure of the app state and hide it from views. Imagine that we decide later on to change the internal state structure. We wouldn’t want to go over all the views in our app and refactor them. Passing through a selector will allow us to confine the refactoring to the only. reducer This is what does our look like: topics/reducer.js The entire current state of our app, including , can be seen . ListView here A few words about “dumb” components is a good example of a “dumb” component. It is not connected to the store nor aware of Redux at all. Unlike the “smart” connected components that are located in , these components are located in ListView /src/containers /src/components “Dumb” components receive data from their parents through props and may hold local component state. Assume you’re implementing a component from scratch. The blinking caret position is an excellent example for local component state that should not find its way into your global app state. TextInput So when do we need to move from a “smart” component to a “dumb” one? Rule: Minimize view logic in smart components by extracting it into dumb components. If you look at the implementation of , you will see it contains view logic like iterating over rows. We want to avoid having this logic in our smart component. This keeps our smart components as wirings only. Another benefit is that the logic is now reusable. ListView TopicsScreen ListView Next scenario — multiple topic selection We’ve completed our first scenario. Let’s move on to the next one — having the user select exactly 3 topics from the list. Our scenario starts with the user clicking on one of the topics. This event is handled by but since this smart component cannot contain any business logic, we’ll dispatch a new action — . This action will also be a , placed in . As you can see, almost every action we export (to be dispatched by views) is a . We usually only dispatch plain object actions from within a thunk in order to update the reducer state. TopicsScreen selectTopic thunk topics/actions.js thunk An interesting aspect about this thunk is that it needs to access the state. Notice how we keep the rule that every state access goes through a selector even here (although some may claim it’s going a bit too far). We’ll have to update the reducer to handle the action and store the new selected topics. There’s an interesting question whether needs to be a at all. Alternatively, we could make a plain object action and move this business logic to the reducer itself. This is a valid strategy. Personally I prefer to keep the business logic in thunks. TOPICS_SELECTED selectTopic thunk selectTopic Once the state updates, we need to propagate the topic selection back to our view. This means adding the selected topics in . Since the view needs to query whether every is selected or not, it is more convenient to pass this data to the view as a map. Since the data has to go through a anyways, this will be a great place to do the transformation. mapStateToProps rowId selector After implementing the above, and refactoring the background color change due to row selection into a new dumb component — — our app looks like . ListRow this A few words about business logic One of the goals of a good methodology is achieving proper separation between views and business logic. Where was our business logic implemented so far? All business logic was implemented under Redux in the directory. Most of it was inside in and some it was inside in . This is actually an official rule: /src/store thunks actions.js selectors reducer.js Rule: Place all business logic inside action handlers (thunks), selectors and reducers. Navigating to the next screen — the posts list When we have more than one screen we need a way to navigate. This is usually achieved using a navigation component like . I want to deliberately avoid using a router in order to keep our example simple. Opinionated external dependencies like routers tend to draw attention away from the conceptual discussion of methodology. react-router Instead, let’s add a state variable , , telling us whether the user completed topic selection or not. Once the user selects 3 topics, we will display a button that once clicked — will finalize selection and move to the next screen. Clicking the button will dispatch an action that sets this state variable directly. selectionFinalized This is all fairly similar to what we’ve been doing so far, the only interesting part is knowing when to display the button (as soon as at least 3 topics are chosen). We may be tempted to add another state variable for this purpose, but this variable can actually be derived from data we already have in the state now. This means we should implement this business logic as a : selector The full implementation of the above is available . In order to do the actual screen switch, we’ll need to change into a connected component and have it listen on in its . The full implementation is available . here App selectionFinalized mapStateToProps here The posts screen — once again state first Since we’re now well experienced in the methodology, we can run through implementing the second screen a bit faster. This new screen deals with a new domain — . In order to make our app modular as possible, we’ll give this domain a separate and separate app state. posts reducer Reminder — the screen’s purpose is to display a list of posts that can be filtered according to topic. The user can click on a post in the list and see its content. Following our structuring , this would work: tips And our new reducer is . posts born First scenario — showing the posts list without filter As usual, when our state is modeled, we move to a simple user scenario and implement it from start to finish. Let’s start with showing the full post list without any filter applied. We need a new smart container to show the posts, we’ll call it and have it dispatch a new action called when it mounts. The action will be a under our new domain in PostsScreen fetchPosts thunk posts/actions.js This is very similar to what we did before, the implementation is . here At the end of the thunk we dispatch the plain action that carries the posts to the reducer. We’ll have to modify our reducer to store the data. In order to show the list in , we need to hook up its to a selector providing this part of the state. We can then display the list by reusing our component. POSTS_FETCHED PostsScreen mapStateToProps ListView Nothing new as well, the implementation is . here Next scenario — filter the post list This scenario starts with showing the user the available filters. We can pull this data from the reducer state using an existing selector. When a filter is changed, we will dispatch an action that will change it directly in the reducer. topics posts The interesting part is applying the filter to the post list. In our app state we currently hold all and the . We don’t want to hold the filtered result in the app state as well because it can be derived from them. Business logic for deriving data runs in right before arriving at the view in . Our selector therefore will be: postsById currentFilter selectors mapStateToProps The full implementation for this stage is available . here Last scenario — showing post details This scenario is actually the simplest one yet. We have an app state variable holding the . All we have to do is update it when the user clicks on a post in the list by dispatching an action. needs this state variable in order to show the post details, which means we’ll need a to drive it in . currentPostId PostsScreen selector mapStateToProps Take a look at the detailed implementation . here And we’re done! This also wraps up the implementation of our entire example app. The full source code of the app is available on GitHub: https://github.com/wix/react-dataflow-example Summary of our opinionated workflow rules App state is a first class citizen, structure it like an in-memory database. Smart components are not allowed to have any logic except dispatching actions. Smart components should always access state through selectors. Minimize view logic in smart components by extracting it into dumb components. Place all business logic inside action handlers (thunks), selectors and reducers. Services must be completely stateless. Remember, Redux offers a lot of room for personal style. There are many alternate workflows with different sets of rules. Some good friends of mine come to mind who prefer instead of and like to place all of the business logic inside only. redux-promise-middleware thunks reducers If you want to share a different methodology that works for you, feel free to PR your own implementation for the above project and we’ll provide it as a branch for comparison.

Redux Step by Step: A Simple and Robust Workflow for Real Life Apps

Machine is kind of magic right? But is it the kind of magic that can make us rich? And I don’t mean lucrative consulting gig rich, I mean digging valuable metals out of the ground rich. The kind of rich you can horde in treasure chests. learning Also I’d been meaning to try out some transfer learning and looking around for a good topic to try it on. Transfer learning is where you take a pre-trained convolution (or other) and use it for your task. It’s great for convolution networks particularly because they take a long time to train. South Australia where I live has taken a lot of effort to share data sets, including some great geological resources, so I thought why not stick the two together and use convolution networks to find gold. The easiest way to make use of convolution networks is to turn them into feature extractors, which is what I’ll show you in this post. network The code that goes along with this blog post can be found on GitHub: _DeepGold using convolution network features to learn mineral data_github.com scottvallance/DeepGold Minerals So to have a computer find you treasure, you’ve got to give it something to work with. Like a treasure map. So my first assumption is that a map of geological features might be enough to spot likely places if you knew how to look at it. Of course it’s not like geologists don’t do this already. They probably even know the names of the minerals and whatnot’s they are looking for unlike me. So I found some papers written on finding minerals (gold in particular) using images. Instead of my first idea of maps, which would have been a fun challenge for a convolution network, there is a multi-spectral satellite image source called ASTER which can provide clues to mineral resources on and in the ground. _ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) is a high resolution imaging instrument that is…_asterweb.jpl.nasa.gov ASTER: Advanced Spaceborne Thermal Emission and Reflection Radiometer Multi-spectral imaging is taking a picture with a camera that can see more than just red, green and blue. Some of these bands in ASTER are good at spotting particular minerals, or mineral families. This following paper goes into how these bands have been used previously and how they could be used. _Remote sensing plays an important role in mineral exploration. One of its proven applications is extracting and…_www.researchgate.net Detecting areas of high-potential gold mineralization using ASTER data While the paper comes up with their own scheme, they mention the following mineral indexes as being particularly good for finding gold: “OHI is the index for OH-bearing minerals, KLI is the kaolinite index, ALI is the alunite index, and CLI is the calcite index”. ASTER has a bunch places where we can download it free of charge, but the downside of wonderful real data set like ASTER is that it’s full of all sorts of real world complications. Like clouds getting in the way, and like the fact that since 2008 some of the sensors on the satellite are basically cooked. Fortunately Geoscience Australia have done a lot of the hard work of stitching and manipulating the data and produced a bunch of big ‘.tif’ images to work with. _This Australian ASTER Initiative, led by CSIRO's Western Australian Centre of Excellence for 3D Mineral Mapping…_www.ga.gov.au National ASTER Maps - Geoscience Australia You can find the data on their ftp site with the links in the Github repo for this post. I grabbed a bunch of images for the location I was interested in, using the AlOH, MgOH and Kaolin band images. That’s only half the story though, we have a lot of images but no clue where the gold is. The easiest way to use machine learning to help us find gold is to give it a bunch of images that have gold in them and a bunch that don’t and get it to figure out the difference. To get that training data we need to know where there’s gold. Another government department the rescue, this time a South Australian state department which has a great little tool they call SARIG. _Edit description_sarig.pir.sa.gov.au South Australian Resource Information Geoserver - Map It’s a map with a lot of different mineral related overlays, one of them particularly interesting to us — ‘All Mines and Mineral Deposits’ with a handy option to download the shape file geo database we have everything we need data wise. Data Processing Now we have lots of data, but it’s not simply a matter of feeding it into computer. You have to slice and dice it into the formats needed. To work geospatial data their are a bunch of great tools collectively known as GDAL: Geospatial Data Abstraction. _Traditionally GDAL used to design the raster part of the library, and OGR the vector part for Simple Features. Starting…_www.gdal.org GDAL: GDAL - Geospatial Data Abstraction Library These tools will happily export info out of our ‘All Mines and Mineral Deposits’ shape files and slice up the massive image files that are also geospatially aware. As each mineral deposit entry in shape file is a point location, we can cut up the large input images into segments around each point of a size appropriate to feed into a convolution network. These slices can have multiple minerals and we don’t allow overlapping slices so that the eventual machine learning doesn’t just get to remember what different segments look like and learn from rote. Then it’s a matter of merging the images from the different bands into a false colour image. A false colour image is when we use something other than real red, green and blue values in image — in this case we’re using the AlOH content, the MgOH content and the Kaolin index. One measure for each colour band. Here’s what a false colour image ready to feed into our convolution network looks like: A false colour image using GEOScience Australia reprocessed ASTER data Convolution Networks Convolution networks have shown amazing ability to learn answers to visual problems. Recently there’s a been a bunch of open source tools released to develop convolution networks including: TensorFlow, Theano, Caffe, Torch. All great tools but we really need something with easy access to pretrained data. For that problem, and for abstracting convolution networks a step away from their implementation, Keras has been incredibly useful. _Keras: Deep Learning library for Theano and TensorFlow_keras.io Keras Documentation The author also provides this with models and pretrained data to work with. The has produced good results, so that’s an obvious starting point for the network. We’ll be using this network to transform our training images into a vector features. Essentially we take the output of the ResNet50 at a point before it turns into a bunch of image class predictions and use that as the input for another classification problem. While training a convolution network is slow using it to generate features is quick and gives us a data source that has been proven to transfer to other problems well. It’s definitely an open question how well it transfers to something as esoteric as a processed false colour satellite image but trying it out is the easiest way to learn. Github repo ResNet50 network Classification with ResNet50 Features After we turn our false colour images into features we can run a bunch of different machine learning algorithms. For this we make use of Spark, a cluster focused graph computation framework with a number of built in machine learning algorithms. _Apache Spark is a fast and general engine for big data processing, with built-in modules for streaming, SQL, machine…_spark.apache.org Apache Spark™ - Lightning-Fast Cluster Computing While the size of the problem doesn’t necessitate using a cluster to solve it, it does provide a nice interface to the algorithms. We can make use of Logistic classification, Support Vector Machine classification, Naive Bayes classification, Random Forests and Multilayer Perceptron classification with just a few lines of code. To make our data ready for Spark we turn it into LibSVM format and then further simplify it by filtering the examples into those containing gold and those that don’t. This leaves us with a lot more not gold examples, so we over-sample the gold ones to give us even numbers because it helps to train an algorithm with balanced data. We load up pyspark and use a few lines of code to run some different classifers. Here are the results of Spark’s classification algorithms: Logistic Regression:Test Accuracy = 0.767756482525 SVM:Test Accuracy = 0.808342728298 Naive Bayes:Test Accuracy = 0.616685456595 Random Forest (10 trees):Test Accuracy = 0.727171 Multilayer Perceptron (layers [2048, 512, 2]):Test Accuracy = 0.775255 This is just a measure of how often the algorithm was right about an image containing gold or not over the total number of testing examples. An accuracy of 80% doesn’t sound bad, and does match with other results shown in the literature but it’s difficult to interpret this data from a simple accuracy measure. We haven’t over sampled the gold examples in the testing data, so they contain a lot more not gold than gold. If an algorithm gets pessimistic about predicting gold it can still do quite well while not telling us very much about where to dig. What we really need is a measure of how likely we are to strike gold if we do go out with a shovel. This is called the precision of the prediction. The flip side of an algorithm being really precise is that it may choose to only predict gold when it’s very sure and miss a lot of examples. The measure of the amount of examples we correctly predict as gold out of the total gold samples is called the recall. We’re probably fine with a low recall, but we’d like precision to be as high as possible. Here’s the precision and recall for the data above: Logistic Regression:Gold Recall = 0.324840764331Gold Precision = 0.337748344371 SVM:Gold Recall = 0.235668789809Gold Precision = 0.425287356322 Naive Bayes:Gold Recall = 0.68152866242Gold Precision = 0.269521410579 Random Forest (10 trees):Gold Recall: 0.575163398693Gold Precision: 0.328358208955 Multilayer Perceptron (layers [2048, 512, 2]):Gold Recall: 0.317880794702Gold Precision: 0.335664335664 So for the purposes of getting out there with a shovel the Support Vector Machine classification really wins. If it says there’s gold then there’s a 42% chance it’s telling the truth, which is out of 17% of samples in the testing data actually having gold. This is just for segments of South Australia that have known mineral deposits though, which is a factor to consider as it might correlate to certain features that aren’t seen in other areas. ‘X’ Marks the Spot So given the data we have is where do we go digging? Obviously not where gold is already labelled because it’s already dug up. Which really just leaves where it isn’t labelled, which has the down side of already being mined for something — but not gold. We’ll take it though, a lot of the data in the mineral deposits is old and abandoned. The Support Vector Machine performed best, but we want to move beyond classification into probabilities. If we clear the threshold that turns the SVMModel in Spark into a classifier we can get raw scores straight from the estimates. Taking the max of the ones where there is no gold gives us this image: Weird boomerang shape marks the spot Let’s check it on the map: Our great mining hope Someone grab the shovels, we’re off! Turns out this is right next door to the White Dam Gold Mine. Which is I guess a good sign, but the mine is actually in our data already it’s just a bit to the left and down. There’s a good chance that the spot we found looks a lot like the mine next door. A quick look through the rest of the likely spots shows a lot of regions next to existing mines. Probably a good thing, but sort of hard to make use of. Running through the maps without mineral data section by section might yield something but it’s a bit speculative and might best be left to the reader. So we didn’t really find gold, but I’m still pleasantly surprised by the accuracy of convolution network features on such a different problem to that which they were trained. Conclusion and Other Projects After searching through the maps with the algorithms we’ve developed a great next step would certainly be to train a convolution network on raw ASTER data. Convolution networks are getting to human levels of success (and over) on certain vision tasks and I think it they could have a big advantage with multispectral data. Humans can only interpret visual data through at most (theoretical not withstanding) three channels but convolution networks are not similarly constrained. The current band ratio used for finding minerals would certainly map well to a network architecture. tetrachromats If you find this article interesting you can . find me on linkedin

Deep Gold: Using Convolution Networks to Find Minerals

We held a team offsite in New York last month. If you know any of us at Spark, you’d guess that we spent most of the time debating with each other, like a family where everyone argues passionately but loves each other at the end of the day. (I‘m guessing most of us at Spark grew up in a family like that. I certainly did.) While we were all together, one topic we spent a lot of time discussing was how digital startups can build long-term competitive barriers. My partner Jeremy Philips led the conversation, and you can read more about his thoughts in a piece he published in the New York Times in August ( ). here A couple of us also applied the thinking that came out of our offsite discussion to hardware. The main thing we all agree on: hardware startups build true competitive barriers, but they have to be careful to avoid simply creating a “toaster.” can When we say a toaster, we mean a product that the market demands, but where margins get eaten away over time due to low barriers to entry and fierce competition. This analogy was popularized by Professor Bruce Greenwald at Columbia Business School (and mentioned by Jeremy in the New York Times article linked above). It’s worth pausing for a minute on this. Some people think that highly-differentiated products will give a company a long-term competitive advantage. But without barriers, we’ve seen that all useful differentiation gets copied in the long-term. Today, intellectual property creates a true barrier in hardware, where copycats emerge out of thin air after a funding announcement. Highly differentiated product ideas are what drive initial customer interest in hardware startups, but competitive barriers are necessary to build a valuable business in the long-run. rarely This subject is important to us at Spark, where we invest very actively in hardware startups. We led Series A rounds for Oculus, Cruise, Thalmic Labs, Superpedestrian, Freight Farms, and many more. We’re strong believers in the potential for hardware startups to create magical new product experiences, make a big impact on society, and generate venture scale returns for their investors in the process. So: for a high-growth hardware , how do you create competitive barriers and not end up as a toaster company? startup From our experience at Spark, the best hardware companies use superior hardware execution as a trojan horse for building a long-term relationship with their customers . Here are three successful strategies we’ve seen. through software and data-driven experiences Oculus built a strong developer community early on in its journey to developing a commercial VR headset, giving the company a meaningful head start against competitors in terms of available content, games, and other applications. The company would never have gotten to where it did without thousands of developers building demos, experimenting, and proselytizing for the brand. The iPhone is a beautiful piece of hardware, but consumers are initially drawn to the platform and ultimately become locked-in because of iOS applications. Similarly, developers are drawn to the Apple customer-base and choose to create applications for iOS. As a consumer, switching from iOS to Android is more than buying a new phone — it’s also starting from scratch on your applications. As a developer, once you have a meaningful customer base on iOS, you will continue building for the platform for a long time. Build a vibrant developer community/ecosystem. . Cruise Automation built an incredibly sophisticated brain for an autonomous vehicle, connecting external sensors (e.g. LIDAR, cameras) and a variety of other data sources to the mechanical automobile drive-train. More importantly, they built machine learning technology to recognize a wide variety of different objects and events. This dataset and the self-training model they developed to improve accuracy over time with use will be very difficult for competitors to replicate, especially since Cruise had a head start. 23andMe built the first direct-to-consumer brand in genetic testing. In my opinion, the real value doesn’t lie in their elegant spit cup design and the core IP around their genetic testing technology. The magic is in the dataset, which 23andMe can continuously mine and leverage on behalf of their growing user-base for medical and genealogical applications. Create a data moat/network effect . Freight Farms launched its flagship Leafy Green Machine product a few years ago, providing customers a hardware platform for hydroponic plant growing. As the company matures, they continue to release software products through their mobile app Farmhand to help farmers manage every facet of their business: from business planning and sales, to yield measurement and management, to purchasing consumables like seeds and nutrients, to sharing recipes with other growers on the platform. This keeps current customers highly engaged on their platform. Square started with a simple headphone-jack attachment to allow SMB merchants to accept credit card payments from a mobile device. Over time, Square launched a variety of other software products into that existing install-base, like merchant analytics, payroll, and streamlined lending, creating a much stickier business than a pure-play PoS and payment processing company. Release sticky software tools to your hardware users Early-stage isn’t all about focusing on the creation of long-term competitive barriers. First and foremost, we invest in products we love and teams that inspire us. Sometimes it’s not clear how a market will develop or how a business will mature on the startup roller-coaster journey, and that’s just fine. venture capital On the other hand, it’s important for us to invest in companies that have a clear vision for the future and the role they will play in it. So we get especially excited when we encounter a hardware team that’s not only maniacally focused on their product and customers, but also thinking proactively about how to protect their company from competitors in the long-run. We love both investing in hardware startups and talking about them. Please let me know what you think about this piece, and give me a shout at john@sparkcapital.com. Thanks to Ryan Shmeizer , Nabeel Hyatt , and Rachael Horwitz for their useful feedback.

Toasters and roller-coasters: how hardware startups can build competitive barriers

George “GeoHot” Hotz Giving a Memorable Pitch at TechCrunch Disrupt The Emmys this past weekend were foreshadowed by the “Geekies” — otherwise known as — the pinnacle for launching your product. I don’t know of one Founder that I’ve coached over the years who hasn’t secretly dreamed of taking the stage at Battlefield. TechCrunch Disrupt Battlefield pitch This year, more than any of the companies competing, another company launching their was in the spotlight: — a company that claims that by the end of the year, for under $1000 your car (any car) will be able to drive you from “Mountain View to San Francisco (about an hour drive for all non-valleyers) without touching anything.” That’s a pretty ambitious and exciting promise! Sounds like they could have won Battlefield hands down had they been competing! startup Comma.ai Watch the Buzzed-About Pitch Yourself The buzz around comma.ai was as much about the product as as the presenter, , the infamous Hacker who is best known for be the first person to “jailbreak” the iphone. I’ve seen many comments on his pitch and style, some saying they couldn’t stand it, some saying it was brilliant. Here’s my perspective: George “Geohot” Hotz What Worked? — This is probably the best explanation I have heard anyone give on Machine Learning. If you put aside the bit of Geek Speak, what he’s basically saying is in order to get a machine to do something on it’s own, you have to use sensors to take in information and then “actuators” to translate it into “outputs” or actions. So basically you can teach a machine to do anything, like drive itself — if you put enough data in. “Let’s have the world teach us to drive.” Simple Solution — Simple is Smart! Take complex concepts and break them down so anyone can understand them. (He could have done without some of the jargon but it was done well) Lesson Learned Look Ma, no Hands! — He goes on to show “chffr” — their Android app (and later the iPhone version dash) that basically is a dashcam app which has recorded a bit over 700 people (That’s not a big number…) driving hundreds of thousands of miles and teaches “their” self driving car to drive, from a phone — no expensive hardware. (And here’s his first company bashing — Uber — more on that later). Towards the end He also showed a prototype of their actual self-driving car device, the comma.ai, which basically replaces your rear view mirror (self installed) and drives instead of you. Though it was a rough mock up and sketch, it got the response he wanted. I will bet you that every geek in the room was salivating at the prospect of having a self driving car by the end of the year… Easy to Understand Demo — Show, don’t Tell — Let us see what a product looks like and imagine ourselves using it. Lesson Learned But somewhere in the middle it all started to go South… What Didn’t Work? — It might have been nerves, or it might have just been copious amounts of hot air but Mr. Hotz was not breathing enough, leaving him gasping for air. That makes the audience feel breathless as well and not in a good way. It almost made him seem like he was on something — in fact the host commented when he finished that she’ll have some of what he’s on — he said “I’m on boredom! I’m bored!” Breathlessness — BREATHE!!! If you’re nervous, your body automatically stops breathing which in turn makes you more breathless and you end up gasping and looking and feeling nervous. Deep, yoga-like breathes at the right moments keep you seemingly calm. And try not to spazz out. Lesson Learned Don’t Bash Your Competition! — First got bashed for using expensive hardware to teach their cars to drive, then it was who was “never going to ship,” then it was who “was hungry but then got millions upon millions of dollars from Uber.” (Yes, they were by Uber for a reported $680M — not bad for a young company…) then — well, he actually said he had the “utmost respect for that company because they actually shipped a self driving car” so Elon Musk emerged rather unscathed from the “Wrath of George.” He went on to say that there “wasn’t nearly enough trash talking in Silicon Valley” and proceeded to bask the “Jokers” — , (Whoaaaaa he doesn’t like them!!! And he continues to bash them in the interview after!!!) , , and the worst bashing of all, who as he said promised to ship a self driving car but then were acquired by GM and he called them a “sellout.” What did he hope to achieve by this? Piss people off? I’m sure that happened — imagine how many investors sitting in the room were invested in these companies? Be memorable? Well, yeah, that worked. Have a bazillion people hear about the fuss and watch his pitch? Yup. Leverage his credibility by knocking the competition? Mmmm nope. Bashing the Competition Uber Google Otto recently acquired Tesla MobileEye Drive.ai Zoox Cruise — Don’t talk trash about your competition — or anyone for that fact. Not only do you run the risk of offending or pissing off people, it takes away a lot of your credibility because instead of thinking how confident you are, they’re probably thinking — “boy if he talks this must trash I wonder what he’s saying about me?” Lesson Learned Watch George’s Interview — a Different Person! What’s amazing to me is in the interview with after the pitch, he seems like a nice guy, a sweet, kinda shy kid that gives solid answers to questions — a bit more bashing done, but had I not seen the pitch I would have really liked this guy. Some people just do better in one on one conversations than in pitches and there are ways to overcome this and pitch with the ease that you convert. So Mr. Hotz, I have a feeling you will be in the public eye quite a bit in the next few months while you polish the UX of your product, I’m happy to help polish the UX of your pitch. :) Darrell Etherington is a Storyteller for Startups who, over 15 years, has around the world raise hundreds of millions of dollars. Need a pitch deck? . Donna Griffit helped hundreds of startups and VC’s Click Here

Stories, News & Mentions about 4DSENSOR

4dsensor’s Stories on HackerNoon

4dsensor's News & Mentions