Agnostic

agnostic

Although we at TNW love new tech, we’re also a bunch of cynics that view it with a certain scepticism (I mean we’d be lousy journalists if we didn’t). We, like you, know what practices we consider bad — such as irresponsible handling of personal data — but it’s harder to pinpoint what’s . good That’s why I asked Torben Friehe, co-founder of the , a European forum for piloting technological advances, what exactly is _good_technology? Good Technology Collective “I don’t know.” Well, that was somewhat underwhelming. (But worry not, I only chose this as the first quote for dramatic effect and to remind you that nothing in life is easy — the truth is that this article is actually worth reading.) The GTC was founded to bring together leading figures in tech from different disciplines — e.g. science, politics, journalism, etc. — to offer counsel on the societal impact of new tech, such as AI, data surveillance, and IOT. So how come Friehe can’t simply say what good technology is? “The thing is that the GTC is founded on seeking the answer to that question. We don’t just want perspectives we agree with, we want to foster real discussion on which direction we want to take tech. The answer could be that technology is bad in principle and we should go back to our caves, I don’t know,” says Friehe and adds laughing “if someone convinces me that’s the right approach, I might live in a cave.” all But jokes aside, Friehe has strong opinions on what good tech is, even though he acknowledges that he alone doesn’t hold all the answers. Intentions, purpose, and accountability (you know, all the fun stuff) “Whether technology is good or bad depends on the people that create it — and I think that’s one of the things that’s neglected. Technology doesn’t appear out of thin air, there are people behind that create it. These people are the ones deciding whether the tech has positive or negative qualities,” says Friehe and emphasizes that engineers also need to become more deeply involved, and question the purpose of the tech they contribute to. Friehe calls for increased accountability among engineers, not just the ‘decision makers’ that form the original intentions or the purpose of a product — whether it’s military tech or data products. “There are a lot of people that are working on technology that don’t even know about the intention behind the product they’re developing. And that’s particularly the case with, and always has been, with engineers.” In Friehe’s opinion, engineers working on frontier technologies are too free to approach their task with almost a child-like zealousness. They can focus solely on the engineering challenges and the joys of solving them, without questioning or even thinking about the end-use of the tech. “The intention of the product is rarely influenced by the engineer. The engineer receives directions and executes them, but we need to change that,” says Friehe. The entire development chain of tech needs to be more interactive and influential and the first step towards that is having engineers have more say. Technology with a purpose — and built with restraint Tech is developing so fast that there are hardly any limits to what we can do, that’s why need to create the restraints — because computing power doesn’t anymore. In Friehe’s opinion, the best way to do that is to educate engineers in ethical related issues and give them more power in forming the purpose of the product. we “The input that engineers give on designing certain solutions influences the design, for example the extent of data collection. Engineers, at least at my company, have a lot of influence on what data we should or shouldn’t record,” says Friehe. The reason for that is that the purpose of the product will ultimately guide where we employ restraint in tech. “It might be a stupid analogy but if you’d collect all the paper documents that you get at home — and literally not throw away anything — your apartment would be flooded with letters.” Friehe feels that engineers of data products need to think about data collection in a similar way. We have the capacity to fill our apartment with letters, but it’s not a smart thing to do, especially if it can be harmful to other people. Although Friehe doesn’t mention it explicitly, I can’t but help but to think about purpose as a reaction to the mentality of advancing tech for tech’s sake. Of course it can be argued that uninhabited innovation has given humanity great gifts, but that’s an attitude that better suited in academia. The reason is that businesses in the private sector (where most technological innovation happens nowadays) always have an underlying and inherent purpose. Whether it’s creating wealth for shareholders or ‘disrupting’ a market, the purpose is there — so it’s important that everyone that participates in making it a reality is a aware of it. So for me, technology might frankly be as simple as technology for the sake of people. But I bet that corporate big wigs could find a way to twist that into something ugly and, for example, argue that people need tech overlords to monetize their personal data to provide them with a platform to argue with their racist uncles. good However, Friehe and the members of GTC are bigger people than me and recognize the need to include all perspectives to reach a meaningful conclusion — because no one group can dictate tech. “There is no single authority that can determine which technology is good or bad,” says Friehe. “And to be clear, it should never be the GTC’s role to determine whether particular tech is good or bad. It should be a forum to discuss what individuals and organizations is good or bad.” think The first step in answering what’s good tech “One of the objectives of the GTC is identifying the threats to good technology. Of course I have an opinion there, but I need to find out whether it’s valid or not. The problem is that people don’t talk about these things, maybe sometimes at pubs — but unfortunately, most influential tech people don’t spend that much time in pubs.” That’s why Friehe and his co-founder Yann Leretaille formed the GTC along with its , which includes influential people in different areas of tech, such as: Expert Council Luciano Floridi, professor of ethics of information and the head of the Oxford Internet Institute Anke Domscheit-Berg, member of Germany’s parliament with the Die Linke party and activist with ties to WikiLeaks Annie Machon, former MI5 counterintelligence officer, data activist and whistleblower Rachel Coldicutt, CEO of Doteveryone, a London-based think tank fighting for a fairer Internet Gerd Leonhard, a top-rated futurist, author and strategic advisor Matt McMullen, founder of RealDoll and Realbotix, the leading AI companionship startup Ida Tin, founder of the internationally acclaimed female health app, Clue Friehe feels that now, for the first time, we actually have the capacity as a society to discuss whether we want new technology, whether we need it, how we want it to be, and what we should do with it. He genuinely believes that if you have a forum to display arguments on good tech — and if you get intelligent people to listen — then people will eventually make up their minds. The GTC will officially open its doors in Berlin on December 15 and you can attend by registering . But despite it only being officially launched now, TNW has been monitoring its inception for the last few months. We’ve even published a few articles by some of their experts — e.g. on , , , and — and it looks like the collective is off to a good start. here algorithms spies and digital privacy problem of disruption AI sex dolls All the original costs of GTC were funded by member donations and in 2018, the collective will operate as an e.V., a nonprofit according to German law, funded through grants and non-corporate donations. But the GTC still needs to be careful with its co-operation with big tech if it wants to make a meaningful contribution to answering what is good tech. Friehe acknowledges that it isn’t just up to him, but the intention is to keep GTC as independent as possible and make sure that future partners cannot influence the agenda of the collective too much. Big tech, just like any other participants, have a right to their opinion on good tech, but it needs to clear that it’s their stance, and not masking it with GTC. Just like you, I’d hoped that this article could’ve given an definite answer to what exactly constitutes as good technology, but it’s too big of a question for now. One thing though is for sure, we’ll never know what good tech is unless we keep asking the question. This story was written by Már Másson Maack , and was originally published on The Next Web. To read the weekly trending TNW story on Hacker Noon, visit Hackernoon.com/tnw .

Technology is Agnostic. The People Who Make it Aren’t.

Companies are now turning to data as one of the most important assets in their businesses, and data engineers are in the midst of managing and improving this asset and its effectiveness. In addition, the integration of data engineering with the use of cloud computing offers scalability, accessibility, and reduced expenses. However, there are security challenges that come with this integration that need to be solved. It is essential to know the rates and causes of cloud security incidents in order to identify the vulnerabilities of the cloud systems. The cost of security incidents in businesses has shockingly increased to 61% in 2024 from the previous year’s 24%. As a data engineer, you are in a position to manage the flow of data in a society that is constantly in evolution. This responsibility is very important in today’s organizations, and if not given due attention, it can affect its overall performance. With the increasing use of cloud technology, it has led to an increase in the frequency of migration processes. It is now evident that cloud security is no more a concern only within the IT department. It has become a crucial factor in the overall business process and productivity. Why is this so important? It is because any wrongdoing or negligence in the system may lead to adverse consequences such as system failures that interrupt business flow, bad image, and breach of data privacy. However, a problem I often see businesses struggle with is that they have their cloud platforms varied and the security tools they use are specific to each ecosystem including the AWS, Azure, or Google Cloud Services. It is possible to become an expert in one system, and this may not allow you to easily transfer your skills to another system. That said, there is a solution to this challenge in the form of an approach that can work on any platform. This means that one can find a number of security principles and practices that are not aligned with any particular vendor to develop a solution that can scale across different clouds. In this piece, I’ll list down the basic principles that a data engineer should know about cloud security and share some insights on the current developments in the field. Key Fundamentals of Cloud Security for Data Engineers Data Encryption Encryption is one of the most effective methods of ensuring that information cannot be obtained by unauthorized persons with ease. When data is stored (At Rest) on the cloud, generally, there are commonly used encryption standards such as Advanced Encryption Standards (AES) with a 256-bit key by the leading cloud service providers. For instance, in AWS, users can simply secure their S3 storage buckets by configuring the encryption settings. To secure data while it’s being transferred from one point to another point (In Transit) on the network, infrastructure upholds TLS (Transport Layer Security) which encrypts communication between services. Access Control and Identity Management The allocation of access control helps in preventing the occurrence of attacks by implementing the notion of least privilege to users and applications and granting them only the required access to perform their tasks on the cloud platforms such as GCP IAM, AWS IAM, or Azure Active Directory where administrators use tools for permission and identity management. The provided example showcases an IAM policy in AWS designed specifically for limiting access to S3 buckets. Network Security The basic infrastructure of cloud security comprises a network system that has features such as Virtual Private Cloud (VPC), network access control lists (NACLs), and firewalls that regulate incoming and outgoing traffic. For instance, in Azure, we can create a Network Security Group (NSG) to filter traffic as shown below. Secure Data Storage and Processing Cloud data must be kept safe while still being easily accessible for processing purposes. Technologies like Bucket Lock in Google Cloud Storage and AWS S3 Object Lock are designed with features that enable immutable data storage in the native environment. Monitoring and Auditing This makes it possible to detect threats in cloud environments since user actions and system events are examined as well as AWS CloudTrail or Azure Monitor logs for any anomalies. Here is a demonstration of how to activate CloudTrail for an AWS account. Compliance and Data Governance Compliance frameworks like GDPR or HIPPA and standards such as SOC 2 require data engineers to incorporate security measures to protect sensitive data adequately. In order to remain compliant with the regulations in place and maintain security standards for information classification and tagging of data, these processes should be conducted by data engineers in collaboration with relevant security teams. As an example of this practice in action, Azure Purview has the capability to automatically categorize data based on the policies that have been applied. Securing Data Pipelines Data pipelines can also be susceptible to man-in-the-middle attacks or incorrect configuration if misshapen by individuals in the middle of a communication channel or if settings are misapplied or misaligned. When choosing among the available candidates for your data pipeline, it is recommended to use either AWS Glue or Apache Airflow as they both provide ways of connecting to your data in a secure manner. For example, the following is a sample setup guide on Apache Airflow which demonstrates the application of an encryption process for enhancing security. Emerging Cloud Security Trends Dealing With an Influx of Data The issue of data volume in cloud environments is a double-edged sword as it creates as many benefits as it does challenges. It’s a well-known fact that the volume of data alone can be quite daunting for tools to manage. The real difficulty, however, does not reside in the capture of data but in the analysis of it - identifying correlations in order to recognize trends and anomalies as well as predicting security threats. Nevertheless, data engineers can employ technologies that go beyond the standard ones and are enhanced by analytical power, Artificial Intelligence, and machine learning. These techniques have the potential to take the currently unstructured data streams and turn them into valuable insights that enhance the overall security posture within the continuously evolving digital environments. Zero Trust Architecture The traditional security model which is based on the concept of perimeter defense is slowly being phased out and giving way to the new zero trust model. The basic tenet of zero trust is based on the premise that no component whether it is a user or a device should be trusted automatically. A major advantage of this methodology is that it focuses on the validation of the user identity, the device, and the access rights that are to be granted. Thus, employing tools that are compatible with various platforms enables data engineers and security personnel to adopt this very effective protective measure in different cloud environments. The Development of Cloud Security Products In a world where cloud services are commonly used and architecture is rather complex, securing systems throughout the systems is not an easy task. This has led to the development of cloud security management tools, which ideally should be able to integrate with other tools to enhance the work of the security teams. The main aim is to enforce the implementation of security policies and gain control and visibility of the different cloud stacks. This should ensure that security measures are put in place irrespective of the technology that underlies it. In Summary Security in the changing world of the cloud is like going on an adventure, and that is a good thing. It is important to deal with security concerns and vulnerabilities when they surface - this puts you in the lead. Zero trust frameworks offer important validations when it comes to handling security issues while the unified security architecture guarantees that policies are well enforced and there is a proper view. But the architectural design of cloud environments is still in the process of evolving, and they need to be adaptable and proactive in order to enhance the shielding. Although all the strategies mentioned here and the ongoing efforts to match up with the advancements can be used to enhance the security of major cloud infrastructure, the threat cannot be entirely ruled out. Companies are now turning to data as one of the most important assets in their businesses, and data engineers are in the midst of managing and improving this asset and its effectiveness. In addition, the integration of data engineering with the use of cloud computing offers scalability, accessibility, and reduced expenses. However, there are security challenges that come with this integration that need to be solved. It is essential to know the rates and causes of cloud security incidents in order to identify the vulnerabilities of the cloud systems. The cost of security incidents in businesses has shockingly increased to 61% in 2024 from the previous year’s 24%. increased to 61% in 2024 increased to 61% in 2024 As a data engineer, you are in a position to manage the flow of data in a society that is constantly in evolution. This responsibility is very important in today’s organizations, and if not given due attention, it can affect its overall performance. With the increasing use of cloud technology , it has led to an increase in the frequency of migration processes. cloud technology cloud technology It is now evident that cloud security is no more a concern only within the IT department. It has become a crucial factor in the overall business process and productivity. Why is this so important? It is because any wrongdoing or negligence in the system may lead to adverse consequences such as system failures that interrupt business flow, bad image, and breach of data privacy. However, a problem I often see businesses struggle with is that they have their cloud platforms varied and the security tools they use are specific to each ecosystem including the AWS, Azure, or Google Cloud Services. It is possible to become an expert in one system, and this may not allow you to easily transfer your skills to another system. That said, there is a solution to this challenge in the form of an approach that can work on any platform. This means that one can find a number of security principles and practices that are not aligned with any particular vendor to develop a solution that can scale across different clouds. In this piece, I’ll list down the basic principles that a data engineer should know about cloud security and share some insights on the current developments in the field. Key Fundamentals of Cloud Security for Data Engineers Key Fundamentals of Cloud Security for Data Engineers Data Encryption Data Encryption Encryption is one of the most effective methods of ensuring that information cannot be obtained by unauthorized persons with ease. When data is stored (At Rest) on the cloud, generally, there are commonly used encryption standards such as Advanced Encryption Standards (AES) with a 256-bit key by the leading cloud service providers. For instance, in AWS, users can simply secure their S3 storage buckets by configuring the encryption settings. To secure data while it’s being transferred from one point to another point (In Transit) on the network, infrastructure upholds TLS (Transport Layer Security) which encrypts communication between services. Access Control and Identity Management Access Control and Identity Management The allocation of access control helps in preventing the occurrence of attacks by implementing the notion of least privilege to users and applications and granting them only the required access to perform their tasks on the cloud platforms such as GCP IAM, AWS IAM, or Azure Active Directory where administrators use tools for permission and identity management. The provided example showcases an IAM policy in AWS designed specifically for limiting access to S3 buckets. Network Security Network Security The basic infrastructure of cloud security comprises a network system that has features such as Virtual Private Cloud (VPC), network access control lists (NACLs), and firewalls that regulate incoming and outgoing traffic. For instance, in Azure, we can create a Network Security Group (NSG) to filter traffic as shown below. Secure Data Storage and Processing Secure Data Storage and Processing Cloud data must be kept safe while still being easily accessible for processing purposes. Technologies like Bucket Lock in Google Cloud Storage and AWS S3 Object Lock are designed with features that enable immutable data storage in the native environment. Monitoring and Auditing Monitoring and Auditing This makes it possible to detect threats in cloud environments since user actions and system events are examined as well as AWS CloudTrail or Azure Monitor logs for any anomalies. Here is a demonstration of how to activate CloudTrail for an AWS account. Compliance and Data Governance Compliance and Data Governance Compliance frameworks like GDPR or HIPPA and standards such as SOC 2 require data engineers to incorporate security measures to protect sensitive data adequately. In order to remain compliant with the regulations in place and maintain security standards for information classification and tagging of data, these processes should be conducted by data engineers in collaboration with relevant security teams. As an example of this practice in action, Azure Purview has the capability to automatically categorize data based on the policies that have been applied. Securing Data Pipelines Securing Data Pipelines Data pipelines can also be susceptible to man-in-the-middle attacks or incorrect configuration if misshapen by individuals in the middle of a communication channel or if settings are misapplied or misaligned. When choosing among the available candidates for your data pipeline, it is recommended to use either AWS Glue or Apache Airflow as they both provide ways of connecting to your data in a secure manner. For example, the following is a sample setup guide on Apache Airflow which demonstrates the application of an encryption process for enhancing security. Emerging Cloud Security Trends Emerging Cloud Security Trends Dealing With an Influx of Data Dealing With an Influx of Data The issue of data volume in cloud environments is a double-edged sword as it creates as many benefits as it does challenges. It’s a well-known fact that the volume of data alone can be quite daunting for tools to manage. The real difficulty, however, does not reside in the capture of data but in the analysis of it - identifying correlations in order to recognize trends and anomalies as well as predicting security threats. data volume data volume Nevertheless, data engineers can employ technologies that go beyond the standard ones and are enhanced by analytical power, Artificial Intelligence, and machine learning. These techniques have the potential to take the currently unstructured data streams and turn them into valuable insights that enhance the overall security posture within the continuously evolving digital environments. analytical power, Artificial Intelligence, and machine learning. analytical power, Artificial Intelligence, and machine learning. Zero Trust Architecture Zero Trust Architecture The traditional security model which is based on the concept of perimeter defense is slowly being phased out and giving way to the new zero trust model . The basic tenet of zero trust is based on the premise that no component whether it is a user or a device should be trusted automatically. zero trust model zero trust model A major advantage of this methodology is that it focuses on the validation of the user identity, the device, and the access rights that are to be granted. Thus, employing tools that are compatible with various platforms enables data engineers and security personnel to adopt this very effective protective measure in different cloud environments. The Development of Cloud Security Products The Development of Cloud Security Products In a world where cloud services are commonly used and architecture is rather complex, securing systems throughout the systems is not an easy task. This has led to the development of cloud security management tools , which ideally should be able to integrate with other tools to enhance the work of the security teams. The main aim is to enforce the implementation of security policies and gain control and visibility of the different cloud stacks. This should ensure that security measures are put in place irrespective of the technology that underlies it. cloud security management tools In Summary In Summary Security in the changing world of the cloud is like going on an adventure, and that is a good thing. It is important to deal with security concerns and vulnerabilities when they surface - this puts you in the lead. Zero trust frameworks offer important validations when it comes to handling security issues while the unified security architecture guarantees that policies are well enforced and there is a proper view. But the architectural design of cloud environments is still in the process of evolving, and they need to be adaptable and proactive in order to enhance the shielding. Although all the strategies mentioned here and the ongoing efforts to match up with the advancements can be used to enhance the security of major cloud infrastructure, the threat cannot be entirely ruled out.

A Platform-Agnostic Approach in Cloud Security for Data Engineers

Authors:
(1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch);
(2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch);
(3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Table of Links Abstract and 1 Introduction 2 Background 2.1 Data Augmentation 2.2 Anchor Regression 3 Anchor Data Augmentation 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.3 Algorithm 4 Experiments and 4.1 Linear synthetic data 4.2 Housing nonlinear regression 4.3 In-distribution Generalization 4.4 Out-of-distribution Robustness 5 Conclusion, Broader Impact, and References A Additional information for Anchor Data Augmentation B Experiments 3 Anchor Data Augmentation In this section, we introduce Anchor Data Augmentation (ADA), a domain-independent data augmentation method inspired by AR. ADA does not require previous knowledge about the data invariances nor manually engineered transformations. As opposed to existing domain-agnostic data augmentation methods [10, 45, 46], we do not require training of an expensive generative model, and the augmentation only adds marginally to the computation complexity of the training. In addition, since ADA originates from a causal regression problem, it can be readily applied to regression problems. Even when ADA does not improve performance, its effect on performance remains minimal. This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch); (2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch); (3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Authors: Authors: (1) Nora Schneider, Computer Science Department, ETH Zurich, Zurich, Switzerland (nschneide@student.ethz.ch); (2) Shirin Goshtasbpour, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (shirin.goshtasbpour@inf.ethz.ch); (3) Fernando Perez-Cruz, Computer Science Department, ETH Zurich, Zurich, Switzerland and Swiss Data Science Center, Zurich, Switzerland (fernando.perezcruz@sdsc.ethz.ch). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Background 2.1 Data Augmentation 2.1 Data Augmentation 2.2 Anchor Regression 2.2 Anchor Regression 3 Anchor Data Augmentation 3 Anchor Data Augmentation 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.1 Comparison to C-Mixup and 3.2 Preserving nonlinear data structure 3.3 Algorithm 3.3 Algorithm 4 Experiments and 4.1 Linear synthetic data 4 Experiments and 4.1 Linear synthetic data 4.2 Housing nonlinear regression 4.2 Housing nonlinear regression 4.3 In-distribution Generalization 4.3 In-distribution Generalization 4.4 Out-of-distribution Robustness 4.4 Out-of-distribution Robustness 5 Conclusion, Broader Impact, and References 5 Conclusion, Broader Impact, and References A Additional information for Anchor Data Augmentation A Additional information for Anchor Data Augmentation B Experiments B Experiments 3 Anchor Data Augmentation In this section, we introduce Anchor Data Augmentation (ADA), a domain-independent data augmentation method inspired by AR. ADA does not require previous knowledge about the data invariances nor manually engineered transformations. As opposed to existing domain-agnostic data augmentation methods [10, 45, 46], we do not require training of an expensive generative model, and the augmentation only adds marginally to the computation complexity of the training. In addition, since ADA originates from a causal regression problem, it can be readily applied to regression problems. Even when ADA does not improve performance, its effect on performance remains minimal. This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

Anchor Data Augmentation (ADA): A Domain-Agnostic Method for Enhancing Regression Models

Authors: (1) William P. Wagner IV, Claremont Graduate University. Table of Links Abstract & Introduction Definition of Key Terms Fundamentals of Internet Operation Encapsulation Usage-Based Economic Models Net Neutrality Legal History Researcher Conclusions Areas for Further Exploration & References 3. Fundamentals of Internet Operation 3.1. Hardware Independence. As long as the hardware meets some basic technical requirements to communicate with its neighbors, there are no specific restrictions on hardware or communications medium. The Internet is Hardware Agnostic. All decisions about a given network’s hardware are left to the network administrator or owner to provide the best solution for their application. Copper, fiber, satellite or cellular – that decision is up to the network owner – the Internet does not care. 3.2. Network Independence. As long as the data packets arriving and leaving a network meet some basic technical requirements to communicate with their neighbors, there are no specific restrictions. The Internet does not care who the owner or administrator of a given network is or what topology that network employs – it will route packets based on “Best Effort” routing – the Internet is Network Agnostic. 3.3. Content Independence. As long as the data packets leaving a network meet some basic technical requirements to communicate with their neighbors, they will be prioritized by type not content or source. For example: A home user’s VoIP packet should almost always be more important than an FTP request from a big corporation, except on a subnet of only that company’s FTP servers. The Internet is necessarily Content Agnostic. This paper is under CC BY-NC-SA 4.0 DEED license. available on arxiv

The Agnostic Nature of the Internet: Understanding Hardware, Network, and Content Neutrality

Joining me in this interview is Christopher Goes, Co-founder of Namada, and we're going to discuss the challenges of integrating privacy with asset transfers, including both fungible and non-fungible tokens, and the technical solutions employed to maintain privacy across various asset types. We'll also touch on the role of zk-SNARK technology in enabling private transfers and the significance of making zero-knowledge proofs feasible on edge devices for wider adoption. Please introduce yourself, and tell us what you do. My name is Christopher; I primarily work on protocol and organizational design for Anoma & Namada at Heliax (which often, but not always, turn out to be two sides of the same coin). What were the most significant technical challenges in developing Namada, and how were they addressed? I would say that there were two significant technical challenges in developing Namada: Firstly, the design and implementation of the MASP circuit, which is quite internally intricate and must simultaneously satisfy safety requirements and provide a clear, efficient interface for users. Luckily, much of the work here was done in the past by the Electric Coin Company on Sapling, and we’ve been able to extend Sapling with multi-asset support and asset conversions while preserving all existing features (e.g., the key hierarchy). Second, the design and implementation of the shielded set rewards system must simultaneously satisfy safety and efficiency requirements and provide an incentive reward mechanism to recirculate the collective benefits of privacy back to individual contributors. As far as I am aware, this particular design is unique to Namada, and rewarding shielded assets carries unique technical challenges compared to reward systems in non-private DeFi contracts. Could you delve deeper into the concept of interchain asset-agnostic privacy? To Namada, interchain asset-agnostic privacy means providing privacy as a service to all existing and future assets which live on the interchain in a uniform way which is standardized in order to make users’ lives easier and which allows users of different assets to benefit from inclusion in a shared anonymity set. Privacy is a public good — the more users use privacy-preserving systems, the more privacy other users get — and Namada provides the protocol infrastructure to realize this privacy-preserving equilibrium. How can organizations balance the need for Asset-Agnostic Privacy with other objectives like transparency and efficiency? I do not think that these objectives are at odds with one another. Organizations and all users of Namada are free to disclose exactly what information they would like to exactly who they would like. Privacy in blockchain systems is less like drawing your curtains closed and more like choosing which group chat to send your messages to. Thanks to recent advances in applied ZK cryptography, using Namada for payments is as fast and as cheap as using a non-private chain. Could you elaborate on how Namada's proof-of-stake L1 architecture supports interchain asset-agnostic privacy? Namada supports interchain asset-agnostic privacy primarily through the IBC protocol, originally developed by the Cosmos ecosystem. Using IBC, other chains can permissionlessly connect to Namada, and users with assets on those chains can send their assets to Namada, transfer them privately, and send them back whenever they would like. Can you delve into the technical workings of the multi-asset shielded pool (MASP) circuit? How does it manage to provide privacy across different types of assets? The multi-asset shielded pool (MASP) provides privacy across different asset types by including in each note (UTXO) an asset type and the amount, owner, and other metadata. When transaction validity is checked, the MASP performs a multi-asset balance check, ensuring zero net balance independently across all asset types. Thanks to homomorphic value commitments with unique bases for each asset type, this is possible without any loss of privacy. For further information, see the MASP specification. How do asset-agnostic shielded transfers work, especially when dealing with both fungible and non-fungible tokens? Fungible and non-fungible tokens aren’t really that different in essence — modulo token-specific logic, a non-fungible token is merely a fungible token with a total supply of one unit, and this is how Namada represents NFTs. NFT-specific logic (e.g., breeding CryptoKitties) lives on the chain which issued the NFTs — Namada provides only privacy-preserving transfers, and standardizing the token logic in this fashion allows all fungible and non-fungible tokens to share the same anonymity set and provide maximum privacy for everyone involved. How does the interoperability with fast-finality chains via IBC and Ethereum via a trustless two-way bridge work practically? Practically, interoperability works via consensus verification: when the Namada validator set produces a block, a light client header and proof of this block is relayed to the other chain (say, Osmosis), which verifies it. This header can subsequently verify packets sent from Namada to Osmosis, such as a packet unshielding previously-shielded OSMO and sending it back to a user on Osmosis. The reverse direction works similarly, just with the two chains reversed. What role do novel zk-SNARKs play in enabling these private transfers? Namada relies on recent advances in zkSNARK technology — particularly, the Groth16 proof system — to provide efficient, compact proofs for the MASP circuits, which facilitate private transfers. Without zero-knowledge proof systems, this kind of practical privacy-preserving system would not be possible. Can you discuss the process of generating Zero-Knowledge Proofs (ZKPs) on edge devices and its importance for mainstream adoption? Users want payments which are cheap, fast, privacy-preserving, and accessible to them on the devices they already have — in particular, smartphones. If we want to bring privacy-preserving payment systems into the mainstream, we must meet users where they are, which will require proof generation support on phones. Luckily, lots of work in these directions has been done already, and much of the labor can be shared between projects. What are the next major milestones for Namada, and how do they align with the project's long-term strategy? As I write, Namada is conducting the Shielded Expedition, a massively multiplayer role-playing incentivized testnet, in preparation for the upcoming mainnet launch. Namada’s success depends on protocol correctness, usability, and user understanding, and the Shielded Expedition is designed to simultaneously test the former and facilitate the latter. How do you envision the future of privacy in blockchain technology? In my view, privacy — or programmable disclosure — is table-stakes for real-world usage of blockchain systems. The blockchain ecosystem has done a solid job funding applied cryptography efforts in recent years, which have made a lot of progress, but this has not yet quite materialized in cleanly integrated user-facing products. I think that we’re on the cusp of a phase change as the final technical and product pieces are now put together to make these systems ready for the mainstream.

Navigating Multichain Environments: The Role of Asset-Agnostic Privacy

Today, I decided to discuss languages and the importance of being language/architecture agnostic. Main Firstly, I observe that the backend world is rediscovering itself and criticizes microservices architecture. For example: Amazon Prime recently reverted to a monolithic structure. For me, the key idea is that we shouldn't blindly follow all mainstream trends. Rather, we should carefully decide which architecture to choose. Maybe the optimal decision for you is to either stick with a monolith or strike a balance between monolith and microservices. Following that, Tobi demonstrated that Rails can now be fast and exhibit high performance. Let's delve into it. Ruby on Rails is often regarded as a framework for startups, providing faster development of new features – a reputation that holds. However, when modern Ruby is combined with contemporary Rails, in capable hands, it can deliver speed even in the late stages of a product's life cycle. Now, let's consider PHP. This language has a bad reputation because it has a low border to enter and if you want to achieve fast and cheap product development you can do it with PHP. In my career, I saw that spaghetti code and mutant architecture can be produced even on fancy Golang or old-school C#/Java. Hence, projects with low maintainability can be agnostic to language and architecture. Examples A few years ago I joined a project that was rewritten from Java to Golang (monolith to microservices). However, the team decided to rewrite the code to another language at first and then extract microservices from it. I think that decision was made because Golang is fancy and it's cool to use it and microservices. However, progress was stopped when a few micro-services (like authorization and some other user services) were implemented. So, after some speed improvements from faster user login and increasing database health. We stumped with a huge half-monolith legacy in Golang that was hardly comprehensible by newbies in that project. Also, it was hard to sell continuous extraction to micro-services to businesses to whom a monolithic architecture and Golang would have been sufficient. Additionally, I can provide a positive example wherein a 10+ TB database was split into distinct databases, each serving specific purposes and supporting discrete microservices. Originally, it was a Ruby on Rails monolith, but due to load growth, certain components were extracted and transitioned into Golang microservices. These modifications caused development velocity improvement and added the ability to sustain high load Pieces of Advice Besides, you have to think over the use cases of your product. For example, maybe you want to analyze how your clients operate with your product. Or you need some monthly report that locks database tables for a long time. So, maybe you need to move analytical work to another service or database. And you always have to think and google about ready solutions . instead of reinventing the wheel Looking forward I hope that all mainstreams will be closely inspected before applying it to a real project. Because now, leading opinion can be just wrong and cause a lot of trouble with understanding and maintaining in the future. Conclusion Ultimately, I want to ask you: What will you choose or how will you adapt and work with various concepts of architectures and languages? What will be your choice?

Languages and Architecture — The Importance of Being Language/Architecture Agnostic

Introducing Nanc: A Backend-Agnostic CMS Perfect for Flutter Apps

ন্যান্সের সাথে পরিচয়: ফ্লটার অ্যাপের জন্য পারফেক্ট একটি ব্যাকএন্ড-অ্যাগনস্টিক সিএমএস

Wir stellen vor: Nanc: Ein Backend-unabhängiges CMS, perfekt für Flutter-Apps

Presentamos Nanc: un CMS independiente del backend perfecto para aplicaciones Flutter

Présentation de Nanc : un CMS indépendant du backend, parfait pour les applications Flutter

नैनक का परिचय: स्पंदन ऐप्स के लिए एक बैकएंड-एग्नोस्टिक सीएमएस परफेक्ट

Nanc の紹介: Flutter アプリに最適なバックエンドに依存しない CMS

Nanc 소개: Flutter 앱에 완벽한 백엔드 독립적 CMS

Apresentando o Nanc: um CMS independente de back-end perfeito para aplicativos Flutter

Nanc ile Tanışın: Flutter Uygulamaları için Mükemmel Bir Arka Uçtan Bağımsız CMS

Giới thiệu Nanc: Một CMS phụ trợ hoàn hảo cho các ứng dụng Flutter

介绍 Nanc：后端不可知的 CMS，非常适合 Flutter 应用程序

Learning how to query data from databases is not just for data scientists, it’s an essential skill for developers, too. Fortunately, NoSQL databases are schema-agnostic. You don’t need to be an expert in thousands of combinations of SQL, data types, and databases to master SQL queries. A good SQL agnostic parser will take care of all but impossible complex queries with ease. If you want to build full-stack applications, it’s important to create apps that use data from databases. Databases can store varies amounts of information in any format, such as text, images, or video. These data stores need to ‘talk’ with one another. Here the role of SQL agnostic parsing software becomes crucial, enabling databases to work among multiple systems rather than having to create a single customized solution from scratch that will degrade in effectiveness over time. SQL Agnostic Queries Make It Work As organizations intake and process an incoming flood of data and traffic, intelligent approaches to managing and querying a database need to be utilized. The proliferation of database types via open-source database software has contributed to the complexity. It’s not an Oracle-only world today. But even it were, there are still major syntax differences among even the simplest operations across all the major SQL databases. Even though they all implement the same SQL standard. A SQL agnostic query engine can account for all the differences. Database-agnostic querying is necessary to allow engineers to focus on the data they need instead of the data access mechanisms at work behind the scenes. To understand the power of SQL, it helps to think about how different types of queries function. For instance: Data Definition Language (DDL), Data Manipulation Language(DML), and Transactions Control Languag (TCL). These all work as an abstract version for various kinds of SQL statements that can be executed across databases and data types. Agnostic queries focus on answering the important question of ‘what’, rather than ‘how’ or ‘where’ information is stored. It also helps users retrieve information without being database literate. It makes SQL queries much less complex to write.  Users see a single unified representation of all the data available to them and get answers in an easy-to-use format. This doesn't mean that the data retrieved under these queries is not being managed or maintained properly. Data can be in a relational database, a graph, a foreign primary key value, or a time series.  Changes in the database schema are continuously tracked and recorded as they always have under standard SQL. Developers don’t need to know that ‘behind the scenes’ a good SQL agnostic query engine is tracking: The kind of database that houses the data, such as relational or key-value. The particular database instance such as a shard, or a partition The version of the database being used such as PostgeSQL or Microsoft SQL server. This is the magic and efficiency of using a database access tool that uses SQL agnostic queries. Example: Apache ShardingSphere is an open-source database ecosystem of tools and middleware that makes good use of existing database resources rather than building another new database technology. Apache ShardingSphere Apache ShardingSphere is an open-source ecosystem consisting of a set of distributed database solutions, including 3 independent products, JDBC, Proxy & Sidecar. They all provide functions of data scale out, distributed transaction, and distributed governance functions, applicable in a variety of situations such as Java isomorphism, heterogeneous language, and cloud native. Apache ShardingSphere is an Apache Top-Level project and is one of the most popular open-source big data projects. It was started about 5 years ago and now has 14K+ stars, over 300 contributors, and 5,100 forks in its community. Here is an exmaple of a query using ShardingSphere’s SQLParserEngine access tool: <insert code snippet here> The queries are more accessible for developers or anyone who wishes to extract information from the database since it's in a format that makes sense, rather than being all over the place with no standard for access. Advantages of SQL Agnostic Queries Backing For Unstructured Text The digital world is changing faster than ever before. It's hard to imagine that there could be so much unstructured data in the world, and yet we're barely scratching the surface. The reality is that 80% or more of all enterprise-level information comes from non-document sources, for example, text messages between employees on their phones. This means companies need a new approach if they want access to valuable business intelligence about what matters most - without having an entire research staff dedicated solely towards understanding where and how to access the data as it comes in. Numerous NoSQL data sets can deal with the ordering of unstructured text either as a local element (MarkLogic Server) or a coordinated arrangement of administrations, including Solr or Elasticsearch. A good SQL agnostic tool is like a Swiss Army knife for your data. It can answer the question of what, when and how much by translating unstructured or disparate elements into something useful that will get you closer to an answer than ever before. Managing Complexity Data scientists have persevered in finding ways to solve queries using the power of data for decades. The first step was optimizing databases, which was complicated and time-consuming, done ‘by hand' from memory - but could not handle large multi-table joins without breaking a sweat! Today's sharding solutions can split up huge tables across many databases so they don't get too heavy on any one server farm or database, but it makes queries more complex. NoSQL parsing software excels at hiding that complexity from users who simply want fast, insightful data retrieval. Reducing Cost The NoSQL data sets handle apportioning (sharding) of a single large, unorganized file across servers. This model supports less expensive commodity hardware and software which can result in significant cost savings for organizations. They won’t have to purchase additional server space or storage media with higher prices than is necessary. To Summarize SQL is the de facto language used to access many different databases and data types for all of today’s apps. SQL agnostic queries are now necessary to  write applications for multiple databases and data types. The syntax of SQL agnostic queries is very similar to standard SQL, so it's easy to learn and use. You don't have to switch between languages when you need a new database. It's easier than ever before because there are more tools like ShardingSphere available for making this process simple and efficient. Now, no matter what major changes come up with future versions of your favorite data platform, you'll always be able to keep using it without any issues, thanks to SQL agnostic queries.

SQL Queries: Why You Need SQL-Agnostic Parsing

Day by day, NFTs are making the case for cross-chain technology stronger. At the very least, they make a case for blockchain agnostic NFT marketplaces considerably more compelling for those in the know. As the market has grown, it has expanded across multiple chains from Ethereum, Binance Smart Chain, Avalanche, Solana, and layer two solutions such as Polygon and Immutable X. Putting gas fees aside, while collectors may or may not care which blockchain their is hosted on, those who collect NFTs for their resale value certainly do care how many pairs of eyes can view their collection. It's all about the potential size of the market, and right now, liquidity for NFT collections is highly segmented. It's not hard to imagine that an NFT tied to a single chain might have lower resale potential than one which can capture the interest of the broader industry, especially if the NFT is hosted on a marginally smaller native chain. NFT While the early NFT market and its growth was tied to the Ethereum network, NFTs, much like DeFi applications before them, are now branching out. One reason for the proliferation of NFTs across multiple chains is the inherent scaling problems of the Ethereum network. Unfortunately, attempts to make Ethereum more scalable have, as yet, seen relatively little success, allowing other chains to make moves on its crown. This is not the whole story, however. NFTs have become such an essential aspect of the wider crypto environment that it would seem almost careless for any major blockchain in the sector not to have at least a few good NFT projects in its stable. Cheaper and Faster While Ethereum remains the king, its grip on the market appears to be weakening. Binance Smart Chain is one of the rivals leading the charge, having now surpassed . The fact that BSC is a fork of Ethereum makes it relatively easy for projects to port from one chain to the other - and with far lower fees, some retail investors feel better cared for on the rival chain. However, BSC is not the only contender with which Ethereum must reckon with. 100 million unique addresses Avalanche, which focuses on speed and low transaction costs, is another one of the emerging competitors seeking to capture a share of the NFT market. The blockchain is now attempting to eat into the market share of the big players. This need for speed has also helped fuel the growth of the aforementioned Binance Smart Chain, as well as Solana and layer two solutions, including Polygon. One of the NFT marketplaces that has capitalized on this drive towards faster and cheaper chains has been . The cross-chain, blockchain agnostic marketplace has successfully positioned itself to become the #1 destination for Avalanche NFTs both by users and trading volume. Solana will become the fifth blockchain the market will support, joining Ethereum, Polygon, Avalanche, and Binance Smart Chain. A 6th, 7th, and 8th chain is expected to follow shortly afterward. NFTrade , one of the first-movers in the NFT market, is also extending cross-chain support - albeit at a more leisurely pace. The marketplace which has been at the forefront of the NFT market on Ethereum also supports Polygon and Klatyn. OpenSea While a number of marketplaces remain tied to a single chain, there are increasing numbers going multi-chain. This is a solid long-term strategy for those who wish to future-proof themselves, especially as cross-chain bridges make native network assets much more fluid. What Matters One of the big questions which market analysts must grapple with is how much users care about decentralization and blockchain fundamentals versus cost, transaction speeds, and other concerns. The top chains have all taken different approaches to address these problems. Solana is one of the fastest chains on the market with 400 millisecond block times, but it is also highly centralized. In September, the Solana mainnet was overloaded and even needed to be rebooted. The implication of such an action is clear; a network that can be stopped and restarted by a controlling entity is arguably not a distributed blockchain at all. Despite this, Solana is also growing considerably in popularity, especially in the NFT sector. For years, a prevailing belief in the blockchain sector has been that decentralization is one of the most important factors (if not the most important), but for some users, it seems that this is not the full case. While Ethereum has grappled for years with Eth2 and moving to Proof-of-Stake, other chains are quite happy to take advantage of this gap of capabilities by providing more centralized solutions. NFT Positive, Blockchain Agnostic The incredible growth of the NFT market and its propagation on multiple chains shows the impending need for multi-chain integration. Cross-chain technology is one way to deal with the segmentation of NFTs across multiple chains; blockchain agnostic marketplaces are another. With increasing numbers of chains and options for creators, collectors, and users, NFTs need cross-chain and blockchain agnostic marketplaces. In any case, the NFT market appears to have a truly bright future ahead of it - on many different blockchains. This article is part of The hosted by HackerNoon in partnership with Gaming Metaverse Writing Contest The Sandbox . Submit your today for your chance to win up to $2000. #gaming-metaverse story

Why NFTs Need Cross-Chain Technology and Blockchain Agnostic Markets

For years now, the attention of the world has remained focused on the financial implications of the cryptocurrency revolution. And that makes sense because the crypto market has grown to a valuation of over $2 trillion. But if you remember a few years ago, the financial aspect of cryptocurrencies wasn't what had most people excited. It was the blockchain. Early on, the business world seized on blockchain technology as the true killer app to emerge from the birth of Bitcoin and its descendants. But as time went on it started to become clear that blockchain wasn't the secure data storage panacea that it was made out to be. And that reality has hampered most businesses' attempts to use it in an effective way. And here at , we struggled with the limitations of the blockchain as we sought to build out our RegTech platform. But then, we realized that there was a better way. A way that leveraged the unique security offered by blockchain technology while driving down data storage costs. And even better, it created an ERC20-based token transaction system that ensured auditability for our users' data. Here's an overview of what we're building and why we think it's the future of RegTech. Sekuritance What is RegTech? To understand the ecosystem we're developing, you must first understand the services it's meant to facilitate. As a RegTech innovator, we provide various services that help businesses comply with whatever regulations they're subject to. For that reason, we were looking to offer PCI-compliant data storage, KYC and AML functions, and risk-scoring for businesses. But unlike incumbent providers of such services, we wanted to offer a unified ecosystem that brought our suite of services together with third-party integration opportunities. And that's where the SKRT utility token comes into play. A Blockchain-Agnostic Platform The SKRT utility token is the transaction medium that unites the Sekuritance RegTech Platform. It's how we track all data vault queries, checks, transactions, traces, and findings. It ties our non-blockchain technology stack to an auditable blockchain that guarantees an audit trail for every service we offer. And our customers use it as the primary means of accessing our services. At present, the SKRT smart contract operates on top of the Ethereum ERC-20 protocol to establish liquidity and a trading regime. But we've built the system to be blockchain-agnostic. That means it can run on Bitcoin, Ethereum, or private permissioned blockchains like Hyperledger. We think that's a critical feature – because the Achilles heel of all blockchain projects is forced obsolescence when newer, better protocols emerge. An All-In-One Solution By establishing the SKRT token and its underlying smart contract, we've been able to create a hybrid system we call the SekurSuite™ product suite. It offloads the payment processing and auditing functionalities to the SKRT ecosystem while the underlying data storage and processing take place in a distributed cloud setting. In this way, we were able to leverage commodity hardened data storage facilities to create a PCI-compliant infrastructure, while creating an immutable access trail via SKRT. But as we've mentioned, we had another motivation for tying our platform together with the SKRT utility token. By creating a token economy, we hope to encourage the adoption of our product suite as well as to encourage third-party developers to publish their products within our ultra-secure app ecosystem. That will allow us to grow our ecosystem into a true hub for RegTech innovation that will push the boundaries of what's possible with current technologies. It will also foster the development of successor technologies by bringing the best and brightest RegTech and blockchain development minds under a single roof. A New Identity Data Model We've also created a program called the Partner Network Rewards System. It's designed to award points to individuals and businesses every time their information is processed by us or any of our partner data providers. Then, the individuals or businesses can claim their data profiles and use their token balance for services through our network – or they can withdraw them to use as they see fit. But most importantly, those profiles will allow for user feedback. This means those who claim their profile will be able to have some direct input into how others view their data. For example, they'll have the opportunity to comment on or explain negative information in their profile, as well as highlight positive information. In a world where we have little control over the information available about us online, and where others' impressions of that data shape our destinies – we see this as a revolutionary feature. The Future of RegTech Going forward, we're looking forward to continuing to refine the SKRT ecosystem and we invite interested developers to explore the opportunities it provides. Together, we hope to create the foundations for a new secure data and RegTech infrastructure to meet the demands of the 21st century. And in the process, we want to leverage blockchain to create the true killer app everyone expected it to spawn. We think we're on the right track. And with your help, we're going to prove it. Featured image: WrightStudio / Adobe Stock

Stories, News & Mentions about AGNOSTIC

Categories

Trending Topics

Stories, News & Mentions about AGNOSTIC

Agnostic’s Stories on HackerNoon

Categories

Trending Topics