Abstract

abstract

Introduction Last chapter of this series. If you have followed up along these blog posts you are now fully capable of creating projects using the Object Oriented paradigm. I am glad you have followed all the sections and in this part, you are going to end up what you have started. In fact, now you will learn how to use abstract classes and several other less common feature of PHP such as reflection API and the singleton design pattern. Keep reading and you will see that as per a lot of other web developers, myself included, these resources will let you do that breakthrough you are looking for. Follow the series … This blog post is the sixth part of “ The complete guide to Object-Oriented Programming: Go from procedural programming to OOP expert in ” PHP If you did not read the other parts yet You can check the other blog posts following the links below , , , , and Introduction to Object-Oriented Programming Inheritance and Interfaces in PHP More Interfaces and Polymorphism Visibility and Static keyword Constructor and Magic methods Abstract Classes and Extra bits Table of Content Abstract classes Method chaining Autoloading Anonymous classes Reflection API Singleton Abstract Classes Abstract classes are very similar to interfaces but they have different rules that a web developer need to be aware of. The definition is quite straightforward, basically, an abstract class is a class that contains abstract methods, Abstract methods are methods that have been declared but not implemented in the code. This kind of classes cannot be constructed as an object, in fact, they need to be extended. A ‘normal’ class that extends an abstract one must implement all methods that are inside the parent. Something to notice is that a child class must define the abstract methods with the same or less restrictive visibility, This means that if in the abstract class there is a protected method it can be implemented only as a public or protected. Another rule is that the number and type or required arguments need to be the same as the parameters. abstract

    abstract

    public

    {

        echo

    }

}

class

    public

    {

        return

    }

}

$myOffice

// This command will return “Building cannot be constructed”

$myOffice->getWindowsCount();

// This command will return 12 I have defined the Office class that extends the abstract class . Building I have implemented and changed its visibility from protected to public. getWindowsCount() There are no abstract methods within the class thus, I can construct an object from it. Office I have added a constructor inside the abstract class so that you will be aware of the fact the parent’s constructor is called when the object is instantiated. What is the difference between an interface and an abstract class? To sum up, Consider an interface as an empty shell. It has only signatures of the methods, which means the methods do not have a body. An interface does not do anything is just a pattern, a set of methods that you need to replicate. On the other hands, abstract classes are proper classes. Even though they look like interfaces the have some specific characteristics. You can specify their use by what they do by a given method name. Here is an example that highlights the differences between the two: interface

{

     public

     public

}

class

{

    public

    public

    {

        echo

    }

    public

    {

        return

    }

} In here we are saying all the Building should look like this than is up to the class to change the functionality. Office abstract class Building{public $windows;public function getWindowsCount(){return $this->windows;}abstract public function build();} class Office extends Building{public function build(){echo “Office built”;}} In an abstract class instead, we are saying that all the Building must have . getWindowsCount() Then we say that the method can be different, so we obligate the class to provide the implementation according to its needs. build() Office Extra / Bonus We have almost finished here with this introduction to Object Oriented in PHP, programming Lot’s of information has been shared and I hope you understood all of it (if not just write your question below). However, there are a few things that have been just quickly mentioned and I would love to go deeper and give you a more wide comprehension on this topic. Method chaining Method chaining is an OOP technique that permits the programmer to perform several different but connected actions in a sequential way. Let’s break down this sentence,“Perform different actions“ means calling different methods or functions; “ ” can be translated into actions related to the same object or instance of the class; Perform connected actions “ ” simply suggest that the methods are called one after another. In a sequential way Here is an example that will clarify the concept: class

{

    public

    function

    {

        $this->message = "Foundation poured - ";

    }

    function

    {

        $this->message .= "framing, plumbing, electrical and HVAC completed -";

        return

    }

    function

    {

        $this->message .= "interior and exterior completed";

        return

    }

    function

    {

        return

    }

}

$myHome

echo Here you go, All the method of the class have one trait in common all of them return the object itself via the use of the keyword , Building $this If you need to brush up your memory I have explained what the keyword does in the . $this introduction to Object-Oriented Programming The last command is chaining down all the methods of the class necessary in order to build up our sentence. The outcome of the last command will look like this: “Foundation poured — framing, plumbing, electrical and HVAC completed — interior and exterior completed”. This is so easy that you can implement this to your code straight away without any problem, just be careful to return the object. Autoloading Any class must be defined before you need to use it, forget to do so will result in an error thrown. Autoload alongside PSR-4 standards allows you to define them when a class is required. I am sure that I do not need to say that this is an incredibly important feature. In PHP autoloading is done by the that accept a function as a parameter, spl_autoload_register() This is quite out-topic and just to explain how it works in detail I will need to write another article, so I will just give you an understandable example: Given that the class you want to autoload is within the “classes/” path, here is what you would do. This is more or less what happens behind the scene of most of the PHP frameworks out there. function

    include

}

spl_autoload_register(myAutoloader); spl_autoload_register has two more parameters, the second is a boolean and defines if the need to throw exceptions when the autoload_function cannot be registered, spl_autoload_register the third parameter is boolean as well and if true it will prepend the autoloader rather then attaching it. If an error occurred and the function was not able to find the specified class, it will throw a fatal error. Anonymous classes Wouldn’t be great if PHP allowed the web developer to create and instantiate objects on the fly? The release of PHP 7 included this present to the list of its features. $myAnonymousBuilding

{

    public

    {

        echo

    }

} Cool, right? Notice how easy is to implement this class inline and that it can even accept parameters like a normal class. The code in the example will output the string “a new class has been constructed in Main Street”. The main use of an anonymous class is the case you need to extend a named class. If, for instance, you will need to override properties or methods, instead of declaring it you can just create an in-line implementation in the same workflow you are currently having. Reflection API Reflection is a very useful feature that has been introduced with the release of PHP 5.0 and became a standard later on 5.3. This API permits you to examine PHP elements and fetch information about them at runtime. For me, one of the most useful places to use reflection API is during automated testing, in specific during PHP Unit. For example, when testing the values of private property in classes you can entree the properties via the API and then make assertion according to the test. There are different reflection classes, each of them allows you to inspect different types of variables. Here are some classes: ReflectionClass ReflectionObject ReflectionMethod ReflectionFunction ReflectionProperty Below a quick example of how the ReflectionClass is used: $ReflectionObject

print_r($ReflectionObject->getMethods()); This example will output an array of all the methods available within the Building class. The parameter passed to the ReflectionClass can be of two types, either a string with the name of the class or a true instance of the class itself. Each of the reflection classes has few methods that allow you to retrieve different information about the element you are inspecting. Singleton Any self-respecting web developer will encounter design patterns in its way to master the programming art sooner or later. Those patterns are nothing more than established methods that solve problems in a predefined way. 99% of PHP framework use them in order to increase the performances and the quality of the code. If you have never heard the words design pattern here is your shot. I want to introduce you the easiest among them all. We are going to use again our imaginary city, we have had several etc, Buildings, Offices, Schools What about the city hall? the mayor needs to keep on eye on all these classes and instantiation and method and properties. In a city, there is only one mayor and he works in only one building the city hall. There cannot be more that one city hall in a city. Well… How do we add this rule in programming? Singleton class cityHall{private static $instance = null;private function __construct(){...}public static function getInstance(){if (self::$instance == null){self::$instance = new Singleton();}return self::$instance;}} Here is it, this is an implementation of the Singleton design pattern. How does it actually work? As you can see the visibility of the constructor is set to private it blocks the creation of the object. In order to create an instance of the class, you need to invoke the static method . getInstance() It checks if the instance has been already created if not, it creates a new instance with the command self::$instance = new Singleton(); Otherwise, it just returns the object already created. The result is that in this case, you are always going to have only one single cityHall in all your program. Where to use the Singleton design pattern in a real case? Singleton is very easy but it can also become problematic to deal with, think at this design pattern whenever you must have a single instance of a class. and it has to be one in a really long term. An easy implementation could be when creating a database class to connect a DB to the code, in the majority of the cases you are going to have only one database per project. This is the perfect use for it. Conclusion The full version of “The complete guide to Object-Oriented Programming” in an all-in-one edition is available in a Kindle format on Amazon. If you have followed all the part of this article this has been quite a journey. You have realized the Object Oriented Programming or OOP is less difficult than it may seem from an outsider. The reality is that as web developers we love to use jargon. Properties, paramethers, interfaces, abstract classes, dependency injection. Those words can provide a headache very quickly. However, just keeping on going taking a little bit at a time you will find that apply these new knowledge is not only possible but easier than you thought. What’s the next step? If you understood the basic concept of this article you are ready to learn about PHP frameworks, There are plenty of them out there, and choose one of them can be overwhelming. Below there is a really deep comparison that I wrote that compare 24 of them, you will not find anything similar elsewhere. Guide to PHP frameworks. If you like this content and you are hungry for some more join the Facebook’s community in which we share info and news just like this one!

Abstract Classes and OOP extras

Abstract Digital Key in Primary Music Education It is well-known the difficulties in perception of any information, including musical one, cause strain of the main functional systems in the child’s body. Statistics and practice show that period of learning with standard music grammar is pulled for several years. At early stage of learning up to thirty percent of children lose their interest to the music subject and leave study. This situation is explained by the excessive load arising at the first contact of the child with a complex format learning the piano. In order to protect the health of parents and children in response to COVID -19 our  Neuromusic Lab proposes to replace face-to-face music training with distance deep learning.We would like to share a technological resource that can both reduce the risk of severe disease and simplify the methods of initial music training. Numeric Cognition 3345. 5432. 1123. 3020. 3345. 5432. 1123. 2010.  Ode to Joy Ludvig van Bethoven 3330. 3330. 3512. 3000. 4444. 4333. 3223. 2050. 3330. 3330. 3512. 3000. 4444. 4333. 3554. 2010.  Jingle Bells Formatted lessons give an equally valuable positive result, regardless of the platform, both interactive and online lesson. The above should not be understood as a rejection of generally accepted music teaching  in the classrooms but to protect the health of children and adolescents, online learning technology will be effective and successful. The future of effective learning is combined with pattern recognition technology. Digital Key Technology is designed to deepen our understanding of cognitive abilities and improve the child's memory capabilities. The Project " Reflection " is aimed at studying the somatosensory integration of nerve signals in the main functional systems of the child’s organism : the central nervous system, auditory , muscular , visual , using electrophysiology measurement techniques - ENG, EMG, EEG, Audiometry. Method EEG allows to research the difference of the integrated synaptic signals , to make the comparative diagrams of dynamics of initiated spikes while perception music characters and digital patterns, plus the time of fatigue of synaptic structures and cells - neurons of cortex is determined . Method  ENG - EMG completes one another, joints the visual analyzer with the neuro-motor function of fingers  and proves , from scientific point of view , the development of muscular fatigue in hands depending on the quantity of eyeballs’ fluctuations. Method Audiometry allows to research the increment of sensitivity to the functional sounds ( pure-tone ,high - low pitched, brain waves, beta range) plus the behavioral audiometry. Innovative technology raises the topic of the use of the Digital Key in Primary Music Education and the Music Industry. Read more: https://www.linkedin.com/pulse/digital-key-initial-music-education-stepanov-sergey/

Distant Deep Teaching 

Bitcoin White Paper: A Peer-to-Peer Electronic Cash System [Full Text] By Satoshi Nakamoto

Packaging Steps and Experience Summary of Lentivirus

BLOCKCHAIN: Is it just a hype?

Privacy in Cloud Computing through Immersion-based Coding: Abstract and Introduction

Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Abstract The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. Introduction Digital health interventions (DHI) support patient health monitoring and treatment provision outside of the traditional clinical setting. To assess patient health status and track the effectiveness of remote treatments patients are requested to regularly fill surveys which consist of numerous questions relating to their physical and mental health. However, responding to questionnaires is laborious and might discourage the consistent use of mobile health (mHealth) applications. Recently Oakley-Girvan et al.[13] performed scoping review of mobile interventions and identified application features that impact patient engagement. A high number of surveys were among the features decreasing engagement with the mHealth apps. This is a problem when health intervention outcome is associated with effective engagement with the application[4]. To encourage long-term engagement app developers use gamification[9]. The use of game elements such as progress tracking and rewards has shown to motivate users to be physically active[19] or maintain a healthy diet[3]. However, the introduction of game elements in mHealth surveys context has been less commonly studied. In this work, we investigate the impact of the inclusion of simple game elements in mobile surveys on completion time and the cognitive burden of self-reporting. We hypothesize that gamification can ease the process of survey completion and ultimately facilitate long-term well-being tracking. To test this hypothesis we train a machine learning tool for cognitive load detection. Then, we develop gamified and not-gamified versions of a mobile survey application and conduct a study with human subjects to assess the effort required to complete these surveys using the developed tool. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Abstract The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. Introduction Digital health interventions (DHI) support patient health monitoring and treatment provision outside of the traditional clinical setting. To assess patient health status and track the effectiveness of remote treatments patients are requested to regularly fill surveys which consist of numerous questions relating to their physical and mental health. However, responding to questionnaires is laborious and might discourage the consistent use of mobile health (mHealth) applications. Recently Oakley-Girvan et al.[13] performed scoping review of mobile interventions and identified application features that impact patient engagement. A high number of surveys were among the features decreasing engagement with the mHealth apps. This is a problem when health intervention outcome is associated with effective engagement with the application[4]. To encourage long-term engagement app developers use gamification[9]. The use of game elements such as progress tracking and rewards has shown to motivate users to be physically active[19] or maintain a healthy diet[3]. However, the introduction of game elements in mHealth surveys context has been less commonly studied. In this work, we investigate the impact of the inclusion of simple game elements in mobile surveys on completion time and the cognitive burden of self-reporting. We hypothesize that gamification can ease the process of survey completion and ultimately facilitate long-term well-being tracking. To test this hypothesis we train a machine learning tool for cognitive load detection. Then, we develop gamified and not-gamified versions of a mobile survey application and conduct a study with human subjects to assess the effort required to complete these surveys using the developed tool. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Gamified Surveys and Cognitive Load Detection in mHealth: Abstract and Introduction

(1) Feng Liang, The University of Texas at Austin and Work partially done during an internship at Meta GenAI (Email: jeffliang@utexas.edu);
(2) Bichen Wu, Meta GenAI and Corresponding author;
(3) Jialiang Wang, Meta GenAI;
(4) Licheng Yu, Meta GenAI;
(5) Kunpeng Li, Meta GenAI;
(6) Yinan Zhao, Meta GenAI;
(7) Ishan Misra, Meta GenAI;
(8) Jia-Bin Huang, Meta GenAI;
(9) Peizhao Zhang, Meta GenAI (Email: stzpz@meta.com);
(10) Peter Vajda, Meta GenAI (Email: vajdap@meta.com);
(11) Diana Marculescu, The University of Texas at Austin (Email: dianam@utexas.edu). Table of Links Abstract and Introduction
2. Related Work
3. Preliminary
4. FlowVid
4.1. Inflating image U-Net to accommodate video
4.2. Training with joint spatial-temporal conditions
4.3. Generation: edit the first frame then propagate


Experiments


5.1. Settings
5.2. Qualitative results
5.3. Quantitative results
5.4. Ablation study and 5.5. Limitations
Conclusion, Acknowledgments and References
A. Webpage Demo and B. Quantitative comparisons Abstract Diffusion models have transformed the image-to-image (I2I) synthesis and are now permeating into videos. However, the advancement of video-to-video (V2V) synthesis has been hampered by the challenge of maintaining temporal consistency across video frames. This paper proposes a consistent V2V synthesis framework by jointly leveraging spatial conditions and temporal optical flow clues within the source video. Contrary to prior methods that strictly adhere to optical flow, our approach harnesses its benefits while handling the imperfection in flow estimation. We encode the optical flow via warping from the first frame and serve it as a supplementary reference in the diffusion model. This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames. Our V2V model, FlowVid, demonstrates remarkable properties: (1) Flexibility: FlowVid works seamlessly with existing I2I models, facilitating various modifications, including stylization, object swaps, and local edits. (2) Efficiency: Generation of a 4-second video with 30 FPS and 512×512 resolution takes only 1.5 minutes, which is 3.1×, 7.2×, and 10.5× faster than CoDeF, Rerender, and TokenFlow, respectively. (3) High-quality: In user studies, our FlowVid is preferred 45.7% of the time, outperforming CoDeF (3.5%), Rerender (10.2%), and TokenFlow (40.4%). 1. Introduction Text-guided Video-to-video (V2V) synthesis, which aims to modify the input video according to given text prompts, has wide applications in various domains, such as short-video creation and more broadly in the film industry. Notable advancements have been seen in text-guided Image-to-Image (I2I) synthesis [4, 14, 31, 43], greatly supported by large pretrained text-to-image diffusion models [37, 39, 40]. However, V2V synthesis remains a formidable task. In contrast to still images, videos encompass an added temporal dimension. Due to the ambiguity of text, there are countless ways to edit frames so they align with the target prompt. Consequently, naively applying I2I models on videos often produces unsatisfactory pixel flickering between frames. To improve frame consistency, pioneering studies edit multiple frames jointly by inflating the image model with spatial-temporal attention [6, 25, 35, 46]. While these methods offer improvements, they do not fully attain the soughtafter temporal consistency. This is because the motion within videos is merely retained in an implicit manner within the attention module. Furthermore, a growing body of research employs explicit optical flow guidance from videos. Specifically, flow is used to derive pixel correspondence, resulting in a pixel-wise mapping between two frames. The correspondence is later utilized to obtain occlusion masks for inpainting [19, 49] or to construct a canonical image [32] However, these hard constraints can be problematic if flow estimation is inaccurate, which is often observed when the flow is determined through a pre-trained model [42, 47, 48]. In this paper, we propose to harness the benefits of optical flow while handling the imperfection in flow estimation. Specifically, we perform flow warping from the first frame to subsequent frames. These warped frames are expected to follow the structure of the original frames but contain some occluded regions (marked as gray), as shown in Figure 2(b). If we use flow as hard constraints, such as inpainting [19, 49] the occluded regions, the inaccurate legs estimation would persist, leading to an undesirable outcome. We seek to include an additional spatial condition, such as a depth map in Figure 2(c), along with a temporal flow condition. The legs’ position is correct in spatial conditions, and therefore, the joint spatial-temporal condition would rectify the imperfect optical flow, resulting in consistent results in Figure 2(d). We build a video diffusion model upon an inflated spatial controlled I2I model. We train the model to predict the input video using spatial conditions (e.g., depth maps) and temporal conditions (flow-warped video). During generation, we employ an edit-propagate procedure: (1) Edit the first frame with prevalent I2I models. (2) Propagate the edits throughout the video using our trained model. The decoupled design allows us to adopt an autoregressive mechanism: the current batch’s last frame can be the next batch’s first frame, allowing us to generate lengthy videos. We train our model with 100k real videos from ShutterStock [1], and it generalizes well to different types of modifications, such as stylization, object swaps, and local edits, as seen in Figure 1. Compared with existing V2V methods, our FlowVid demonstrates significant advantages in terms of efficiency and quality. Our FlowVid can generate 120 frames (4 seconds at 30 FPS) in high-resolution (512×512) in just 1.5 minutes on one A-100 GPU, which is 3.1×, 7.2× and 10.5× faster than state-of-the-art methods CoDeF [32] (4.6 minutes) Rerender [49] (10.8 minutes), and TokenFlow [13] (15.8 minutes). We conducted a user study on 25 DAVIS [34] videos and designed 115 prompts. Results show that our method is more robust and achieves a preference rate of 45.7% compared to CoDeF (3.5%) Rerender (10.2%) and TokenFlow (40.4%) Our contributions are summarized as follows: (1) We introduce FlowVid, a V2V synthesis method that harnesses the benefits of optical flow, while delicately handling the imperfection in flow estimation. (2) Our decoupled editpropagate design supports multiple applications, including stylization, object swap, and local editing. Furthermore, it empowers us to generate lengthy videos via autoregressive evaluation. (3) Large-scale human evaluation indicates the efficiency and high generation quality of FlowVid. This paper is available on arxiv under CC 4.0 license. (1) Feng Liang, The University of Texas at Austin and Work partially done during an internship at Meta GenAI (Email: jeffliang@utexas.edu); (2) Bichen Wu, Meta GenAI and Corresponding author; (3) Jialiang Wang, Meta GenAI; (4) Licheng Yu, Meta GenAI; (5) Kunpeng Li, Meta GenAI; (6) Yinan Zhao, Meta GenAI; (7) Ishan Misra, Meta GenAI; (8) Jia-Bin Huang, Meta GenAI; (9) Peizhao Zhang, Meta GenAI (Email: stzpz@meta.com); (10) Peter Vajda, Meta GenAI (Email: vajdap@meta.com); (11) Diana Marculescu, The University of Texas at Austin (Email: dianam@utexas.edu). (1) Feng Liang, The University of Texas at Austin and Work partially done during an internship at Meta GenAI (Email: jeffliang@utexas.edu); (2) Bichen Wu, Meta GenAI and Corresponding author; (3) Jialiang Wang, Meta GenAI; (4) Licheng Yu, Meta GenAI; (5) Kunpeng Li, Meta GenAI; (6) Yinan Zhao, Meta GenAI; (7) Ishan Misra, Meta GenAI; (8) Jia-Bin Huang, Meta GenAI; (9) Peizhao Zhang, Meta GenAI (Email: stzpz@meta.com); (10) Peter Vajda, Meta GenAI (Email: vajdap@meta.com); (11) Diana Marculescu, The University of Texas at Austin (Email: dianam@utexas.edu). Table of Links Abstract and Introduction 2. Related Work 3. Preliminary 4. FlowVid 4.1. Inflating image U-Net to accommodate video 4.2. Training with joint spatial-temporal conditions 4.3. Generation: edit the first frame then propagate Experiments 5.1. Settings 5.2. Qualitative results 5.3. Quantitative results 5.4. Ablation study and 5.5. Limitations Conclusion, Acknowledgments and References A. Webpage Demo and B. Quantitative comparisons Abstract and Introduction Abstract and Introduction 2. Related Work 2. Related Work 3. Preliminary 3. Preliminary 4. FlowVid 4. FlowVid 4 .1. Inflating image U-Net to accommodate video .1. Inflating image U-Net to accommodate video 4.2. Training with joint spatial-temporal conditions 4.2. Training with joint spatial-temporal conditions 4.3. Generation: edit the first frame then propagate 4.3. Generation: edit the first frame then propagate Experiments Experiments Experiments 5.1. Settings 5.1. Settings 5.2. Qualitative results 5.2. Qualitative results 5.3. Quantitative results 5.3. Quantitative results 5.4. Ablation study and 5.5. Limitations 5.4. Ablation study and 5.5. Limitations Conclusion, Acknowledgments and References Conclusion, Acknowledgments and References A. Webpage Demo and B. Quantitative comparisons A. Webpage Demo and B. Quantitative comparisons Abstract Diffusion models have transformed the image-to-image (I2I) synthesis and are now permeating into videos. However, the advancement of video-to-video (V2V) synthesis has been hampered by the challenge of maintaining temporal consistency across video frames. This paper proposes a consistent V2V synthesis framework by jointly leveraging spatial conditions and temporal optical flow clues within the source video. Contrary to prior methods that strictly adhere to optical flow, our approach harnesses its benefits while handling the imperfection in flow estimation. We encode the optical flow via warping from the first frame and serve it as a supplementary reference in the diffusion model. This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames. Our V2V model, FlowVid, demonstrates remarkable properties: (1) Flexibility: FlowVid works seamlessly with existing I2I models, facilitating various modifications, including stylization, object swaps, and local edits. (2) Efficiency: Generation of a 4-second video with 30 FPS and 512×512 resolution takes only 1.5 minutes, which is 3.1×, 7.2×, and 10.5× faster than CoDeF, Rerender, and TokenFlow, respectively. (3) High-quality: In user studies, our FlowVid is preferred 45.7% of the time, outperforming CoDeF (3.5%), Rerender (10.2%), and TokenFlow (40.4%). 1. Introduction Text-guided Video-to-video (V2V) synthesis, which aims to modify the input video according to given text prompts, has wide applications in various domains, such as short-video creation and more broadly in the film industry. Notable advancements have been seen in text-guided Image-to-Image (I2I) synthesis [4, 14, 31, 43], greatly supported by large pretrained text-to-image diffusion models [37, 39, 40]. However, V2V synthesis remains a formidable task. In contrast to still images, videos encompass an added temporal dimension. Due to the ambiguity of text, there are countless ways to edit frames so they align with the target prompt. Consequently, naively applying I2I models on videos often produces unsatisfactory pixel flickering between frames. To improve frame consistency, pioneering studies edit multiple frames jointly by inflating the image model with spatial-temporal attention [6, 25, 35, 46]. While these methods offer improvements, they do not fully attain the soughtafter temporal consistency. This is because the motion within videos is merely retained in an implicit manner within the attention module. Furthermore, a growing body of research employs explicit optical flow guidance from videos. Specifically, flow is used to derive pixel correspondence, resulting in a pixel-wise mapping between two frames. The correspondence is later utilized to obtain occlusion masks for inpainting [19, 49] or to construct a canonical image [32] However, these hard constraints can be problematic if flow estimation is inaccurate, which is often observed when the flow is determined through a pre-trained model [42, 47, 48]. In this paper, we propose to harness the benefits of optical flow while handling the imperfection in flow estimation. Specifically, we perform flow warping from the first frame to subsequent frames. These warped frames are expected to follow the structure of the original frames but contain some occluded regions (marked as gray), as shown in Figure 2(b). If we use flow as hard constraints, such as inpainting [19, 49] the occluded regions, the inaccurate legs estimation would persist, leading to an undesirable outcome. We seek to include an additional spatial condition, such as a depth map in Figure 2(c), along with a temporal flow condition. The legs’ position is correct in spatial conditions, and therefore, the joint spatial-temporal condition would rectify the imperfect optical flow, resulting in consistent results in Figure 2(d). We build a video diffusion model upon an inflated spatial controlled I2I model. We train the model to predict the input video using spatial conditions (e.g., depth maps) and temporal conditions (flow-warped video). During generation, we employ an edit-propagate procedure: (1) Edit the first frame with prevalent I2I models. (2) Propagate the edits throughout the video using our trained model. The decoupled design allows us to adopt an autoregressive mechanism: the current batch’s last frame can be the next batch’s first frame, allowing us to generate lengthy videos. We train our model with 100k real videos from ShutterStock [1], and it generalizes well to different types of modifications, such as stylization, object swaps, and local edits, as seen in Figure 1. Compared with existing V2V methods, our FlowVid demonstrates significant advantages in terms of efficiency and quality. Our FlowVid can generate 120 frames (4 seconds at 30 FPS) in high-resolution (512×512) in just 1.5 minutes on one A-100 GPU, which is 3.1×, 7.2× and 10.5× faster than state-of-the-art methods CoDeF [32] (4.6 minutes) Rerender [49] (10.8 minutes), and TokenFlow [13] (15.8 minutes). We conducted a user study on 25 DAVIS [34] videos and designed 115 prompts. Results show that our method is more robust and achieves a preference rate of 45.7% compared to CoDeF (3.5%) Rerender (10.2%) and TokenFlow (40.4%) Our contributions are summarized as follows: (1) We introduce FlowVid, a V2V synthesis method that harnesses the benefits of optical flow, while delicately handling the imperfection in flow estimation. (2) Our decoupled editpropagate design supports multiple applications, including stylization, object swap, and local editing. Furthermore, it empowers us to generate lengthy videos via autoregressive evaluation. (3) Large-scale human evaluation indicates the efficiency and high generation quality of FlowVid. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis: Abstract and Intro

Authors:
(1) Liang Wang, Microsoft Corporation, and Correspondence to (wangliang@microsoft.com);
(2) Nan Yang, Microsoft Corporation, and correspondence to (nanya@microsoft.com);
(3) Xiaolong Huang, Microsoft Corporation;
(4) Linjun Yang, Microsoft Corporation;
(5) Rangan Majumder, Microsoft Corporation;
(6) Furu Wei, Microsoft Corporation and Correspondence to (fuwei@microsoft.com). Table of Links Abstract and 1 Introduction 2 Related Work 3 Method 3.1 Synthetic Data Generation 3.2 Training 4 Experiments 4.1 Statistics of the Synthetic Data 4.2 Model Fine-tuning and Evaluation 4.3 Main Results 4.4 Multilingual Retrieval 5 Analysis 5.1 Is Contrastive Pre-training Necessary? 5.2 Extending to Long Text Embeddings and 5.3 Analysis of Training Hyperparameters 6 Conclusion and References A Implementation Details B Test Set Contamination Analysis C Prompts for Synthetic Data Generation D Instructions for Training and Evaluation Abstract In this paper, we introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. Unlike existing methods that often depend on multi-stage intermediate pre-training with billions of weakly-supervised text pairs, followed by fine-tuning with a few labeled datasets, our method does not require building complex training pipelines or relying on manually collected datasets that are often constrained by task diversity and language coverage. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across nearly 100 languages. We then fine-tune open-source decoder-only LLMs on the synthetic data using standard contrastive loss. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data. Furthermore, when fine-tuned with a mixture of synthetic and labeled data, our model sets new state-of-the-art results on the BEIR and MTEB benchmarks. 1 Introduction Text embeddings are vector representations of natural language that encode its semantic information. They are widely used in various natural language processing (NLP) tasks, such as information retrieval (IR), question answering, semantic textual similarity, bitext mining, item recommendation, etc. In the field of IR, the first-stage retrieval often relies on text embeddings to efficiently recall a small set of candidate documents from a large-scale corpus using approximate nearest neighbor search techniques. Embedding-based retrieval is also a crucial component of retrieval-augmented generation (RAG) [21], which is an emerging paradigm that enables large language models (LLMs) to access dynamic external knowledge without modifying the model parameters. Source attribution of generated text is another important application of text embeddings [14] that can improve the interpretability and trustworthiness of LLMs. Previous studies have demonstrated that weighted average of pre-trained word embeddings [35, 1] is a strong baseline for measuring semantic similarity. However, these methods fail to capture the rich contextual information of natural language. With the advent of pre-trained language models [11], Sentence-BERT [37] and SimCSE [13] have been proposed to learn text embeddings by finetuning BERT on natural language inference (NLI) datasets. To further enhance the performance and robustness of text embeddings, state-of-the-art methods like E5 [46] and BGE [48] employ a more complex multi-stage training paradigm that first pre-trains on billions of weakly-supervised text pairs, and then fine-tunes on several labeled datasets. Existing multi-stage approaches suffer from several drawbacks. Firstly, they entail a complex multi-stage training pipeline that demands substantial engineering efforts to curate large amounts of relevance pairs. Secondly, they rely on manually collected datasets that are often constrained by the diversity of tasks and the coverage of languages. For instance, Instructor [40] is only trained on instructions from 330 English datasets, whereas BGE [48] only focuses on high-resource languages such as English and Chinese. Moreover, most existing methods employ BERT-style encoders as the backbone, neglecting the recent advances of training better LLMs and related techniques such as context length extension [38]. In this paper, we propose a novel method for text embeddings that leverages LLMs to overcome the limitations of existing approaches. We use proprietary LLMs to generate synthetic data for a diverse range of text embedding tasks in 93 languages, covering hundreds of thousands of embedding tasks. Specifically, we use a two-step prompting strategy that first prompts the LLMs to brainstorm a pool of candidate tasks, and then prompts the LLMs to generate data conditioned on a given task from the pool. To cover various application scenarios, we design multiple prompt templates for each task type and combine the generated data from different templates to boost diversity. For the text embedding models, we opt for fine-tuning powerful open-source LLMs rather than small BERT-style models. Since LLMs such as Mistral [19] have been extensively pre-trained on web-scale data, contrastive pre-training offers little additional benefit. We demonstrate that Mistral-7B, when fine-tuned solely on synthetic data, attains competitive performance on the BEIR [42] and MTEB [28] benchmarks. This is particularly intriguing considering that this setting does not involve any labeled data. When fine-tuned on a mixture of synthetic and labeled data, our model achieves new state-of-the-art results, surpassing previous methods by a significant margin (+2%). The entire training process requires less than 1k steps. Moreover, we empirically validate that our model can effectively perform personalized passkey retrieval for inputs up to 32k tokens by altering the rotation base of the position embeddings, extending the context length beyond the conventional 512 token limit. Regarding its multilinguality, our model excels on high-resource languages. However, for low-resource languages, there is still room for improvement as current open-source LLMs are not adequately pre-trained on them. This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Liang Wang, Microsoft Corporation, and Correspondence to (wangliang@microsoft.com); (2) Nan Yang, Microsoft Corporation, and correspondence to (nanya@microsoft.com); (3) Xiaolong Huang, Microsoft Corporation; (4) Linjun Yang, Microsoft Corporation; (5) Rangan Majumder, Microsoft Corporation; (6) Furu Wei, Microsoft Corporation and Correspondence to (fuwei@microsoft.com). Authors: Authors: (1) Liang Wang, Microsoft Corporation, and Correspondence to (wangliang@microsoft.com); (2) Nan Yang, Microsoft Corporation, and correspondence to (nanya@microsoft.com); (3) Xiaolong Huang, Microsoft Corporation; (4) Linjun Yang, Microsoft Corporation; (5) Rangan Majumder, Microsoft Corporation; (6) Furu Wei, Microsoft Corporation and Correspondence to (fuwei@microsoft.com). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Related Work 2 Related Work 3 Method 3.1 Synthetic Data Generation 3.1 Synthetic Data Generation 3.2 Training 3.2 Training 4 Experiments 4.1 Statistics of the Synthetic Data 4.1 Statistics of the Synthetic Data 4.2 Model Fine-tuning and Evaluation 4.2 Model Fine-tuning and Evaluation 4.3 Main Results 4.3 Main Results 4.4 Multilingual Retrieval 4.4 Multilingual Retrieval 5 Analysis 5.1 Is Contrastive Pre-training Necessary? 5.1 Is Contrastive Pre-training Necessary? 5.2 Extending to Long Text Embeddings and 5.3 Analysis of Training Hyperparameters 5.2 Extending to Long Text Embeddings and 5.3 Analysis of Training Hyperparameters 6 Conclusion and References 6 Conclusion and References A Implementation Details A Implementation Details B Test Set Contamination Analysis B Test Set Contamination Analysis C Prompts for Synthetic Data Generation C Prompts for Synthetic Data Generation D Instructions for Training and Evaluation D Instructions for Training and Evaluation Abstract In this paper, we introduce a novel and simple method for obtaining high-quality text embeddings using only synthetic data and less than 1k training steps. Unlike existing methods that often depend on multi-stage intermediate pre-training with billions of weakly-supervised text pairs, followed by fine-tuning with a few labeled datasets, our method does not require building complex training pipelines or relying on manually collected datasets that are often constrained by task diversity and language coverage. We leverage proprietary LLMs to generate diverse synthetic data for hundreds of thousands of text embedding tasks across nearly 100 languages. We then fine-tune open-source decoder-only LLMs on the synthetic data using standard contrastive loss. Experiments demonstrate that our method achieves strong performance on highly competitive text embedding benchmarks without using any labeled data. Furthermore, when fine-tuned with a mixture of synthetic and labeled data, our model sets new state-of-the-art results on the BEIR and MTEB benchmarks. 1 Introduction Text embeddings are vector representations of natural language that encode its semantic information. They are widely used in various natural language processing (NLP) tasks, such as information retrieval (IR), question answering, semantic textual similarity, bitext mining, item recommendation, etc. In the field of IR, the first-stage retrieval often relies on text embeddings to efficiently recall a small set of candidate documents from a large-scale corpus using approximate nearest neighbor search techniques. Embedding-based retrieval is also a crucial component of retrieval-augmented generation (RAG) [21], which is an emerging paradigm that enables large language models (LLMs) to access dynamic external knowledge without modifying the model parameters. Source attribution of generated text is another important application of text embeddings [14] that can improve the interpretability and trustworthiness of LLMs. Previous studies have demonstrated that weighted average of pre-trained word embeddings [35, 1] is a strong baseline for measuring semantic similarity. However, these methods fail to capture the rich contextual information of natural language. With the advent of pre-trained language models [11], Sentence-BERT [37] and SimCSE [13] have been proposed to learn text embeddings by finetuning BERT on natural language inference (NLI) datasets. To further enhance the performance and robustness of text embeddings, state-of-the-art methods like E5 [46] and BGE [48] employ a more complex multi-stage training paradigm that first pre-trains on billions of weakly-supervised text pairs, and then fine-tunes on several labeled datasets. Existing multi-stage approaches suffer from several drawbacks. Firstly, they entail a complex multi-stage training pipeline that demands substantial engineering efforts to curate large amounts of relevance pairs. Secondly, they rely on manually collected datasets that are often constrained by the diversity of tasks and the coverage of languages. For instance, Instructor [40] is only trained on instructions from 330 English datasets, whereas BGE [48] only focuses on high-resource languages such as English and Chinese. Moreover, most existing methods employ BERT-style encoders as the backbone, neglecting the recent advances of training better LLMs and related techniques such as context length extension [38]. In this paper, we propose a novel method for text embeddings that leverages LLMs to overcome the limitations of existing approaches. We use proprietary LLMs to generate synthetic data for a diverse range of text embedding tasks in 93 languages, covering hundreds of thousands of embedding tasks. Specifically, we use a two-step prompting strategy that first prompts the LLMs to brainstorm a pool of candidate tasks, and then prompts the LLMs to generate data conditioned on a given task from the pool. To cover various application scenarios, we design multiple prompt templates for each task type and combine the generated data from different templates to boost diversity. For the text embedding models, we opt for fine-tuning powerful open-source LLMs rather than small BERT-style models. Since LLMs such as Mistral [19] have been extensively pre-trained on web-scale data, contrastive pre-training offers little additional benefit. We demonstrate that Mistral-7B, when fine-tuned solely on synthetic data, attains competitive performance on the BEIR [42] and MTEB [28] benchmarks. This is particularly intriguing considering that this setting does not involve any labeled data. When fine-tuned on a mixture of synthetic and labeled data, our model achieves new state-of-the-art results, surpassing previous methods by a significant margin (+2%). The entire training process requires less than 1k steps. Moreover, we empirically validate that our model can effectively perform personalized passkey retrieval for inputs up to 32k tokens by altering the rotation base of the position embeddings, extending the context length beyond the conventional 512 token limit. Regarding its multilinguality, our model excels on high-resource languages. However, for low-resource languages, there is still room for improvement as current open-source LLMs are not adequately pre-trained on them. This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

Improving Text Embeddings with
Large Language Models: Abstract and Introduction

Infrastructure as code (IaC) has become increasingly challenging as organization and usage scale. Inconsistent configurations, security gaps, and increasing costs remain to be hard to manage without the right level of abstraction, but what's the right level of abstraction? From DevOps to Infrastructure as Code DevOps started with engineers clicking around the AWS console and writing scripts to automate tasks like creating EC2 instances and setting up VPCs, but as usage grew, Infrastructure as Code (IaC) tools such as Terraform and CloudFormation emerged. Instead of writing shell scripts, infrastructure engineers can write declarative code to define how the cloud resources should be deployed. This is a great step up in terms of abstraction from the one-off automation scripts. On one hand, the declarative nature of the IaC tools makes it easy enough for most developers to write (you can think of it as configuration instead of code); on the other, it's not abstracted enough to give platform engineers the ability to write modularized code for developers to create infrastructure across the organization. Take resource tagging for example, for both cost tracking and governance purposes, it's a very common use case to want to tag cloud resources by application, team, business unit, cost center, etc. With Terraform, application engineers need to manually specify these tags for each resource. Even if platform engineering teams create the relevant terraform "tag" module to enforce common fields, application engineers still need to provide the tag values every time. The declarative nature of IaC makes it hard for platform engineering teams to automatically populate these tag values from a central registry of applications and teams. This becomes especially problematic as the number of resources and size of the organization grows. This led to the next question: how can we create a higher level of abstraction on top of IaC that addresses these enterprise-scale challenges? Creating Platform Abstraction for Scale Creating the right level of platform abstraction enables large organizations to manage infrastructure at scale more effectively and increase developer velocity. Platform engineering teams often use a mix of strategies, depending on use cases, resources, and expertise within the organization. 1. Terraform Modules or Templates Creating reusable terraform modules or templates is a common and easy way to create higher-level abstractions. While flexible, it's a lower-level abstraction, and making organization-wide changes can be time-consuming as they require review and testing by individual application teams. For organizations lacking resources and expertise, vendors like Resourcely create templated terraform code with enforced practice and security. 2. Imperative IaC With declarative IaC, integrating external processes into infrastructure deployment often requires workarounds. Platform teams frequently combine declarative IaC with one-off scripts or API calls in their CI/CD pipelines. Tools like AWS CDK (Cloud Development Kit), CDK for Terraform, and Pulumi allow teams to generate IaC using imperative programming languages, enabling platform engineers to create infrastructure code alongside dynamic processes. While this provides greater flexibility and abstraction, it poses a higher adoption barrier for application teams less familiar with these "CDK" frameworks. 3. Infrastructure and Platform as a Service At the highest abstraction level, platforms like Databricks, Confluent, and Elastic wrap entire infrastructure deployments, exposing only relevant configurations to developers. This allows the provisioning of complex, distributed systems with minimal effort. Kubernetes similarly abstracts compute resource management, enabling developers to create deployments using high-level configurations. While the high-level abstraction allows platform teams to standardize provisioning, governance, and security at scale for supported use cases, it comes at the cost of reduced flexibility and potential vendor lock-in. New or custom use cases can require much longer implementation cycles. Platform Abstraction: Balancing Flexibility and Control Defining the "right" level of abstraction is never easy. Too little abstraction can lead to frequent manual updates and migrations, inconsistent standards, and security and governance implementations. However, too much abstraction can force teams to adapt their use cases to fit the platform's limitations, slowing down developer velocity. The ideal abstraction varies case by case. For real-time fraud detection or experimental features, teams often need flexibility. Templated terraform might provide sufficient guardrails while enabling developers to easily create custom deployments. In contrast, common use cases like data streaming involving Kafka, connectors, and multi-platform integrations typically require more abstraction. Platform teams often create end-to-end deployments, exposing only necessary configs. However, data pipelines can be nuanced, and platform teams often expose lower-level components, allowing application teams to create custom implementations easily and bring back these features to the platform as they mature. For less complicated scenarios like application deployment, most teams only need to specify a Docker image, CPU, and memory requirements. Platform teams can wrap up the CI/CD process without exposing lower-level components. Knowing the different implementation strategies and levels of abstraction for various scenarios, how can we build platforms that scale? Future of Platform Engineering Data and AI Platform as a Service Kubernetes has simplified infrastructure and application deployment, but data platforms have become increasingly complex. The proliferation of specialized data tools, compounded by AI and ML solutions, has made standardization, integration, security, and governance much harder to manage than infrastructure or application deployment. Vendors like Confluent, Snowflake, Databricks, and dbt are improving the developer experience with more automation and integrations, but they often operate independently. This fragmentation makes standardizing multi-directional integrations across identity and access management, data governance, security, and cost control even more challenging. Developing a standardized, secure, and scalable solution for multi-platform environments is now a fast-evolving area for platform engineering teams. Higher Abstraction for Standardization, Security, and Governance While application developers can use low-level terraform to create infrastructure on their own, this approach doesn’t scale at the enterprise level. Standardization, security, governance, and cost efficiency aren't just “nice to have” but "required" for large organizations. Many companies undertake multi-year, large-scale migrations due to a lack of abstraction where it’s needed, meanwhile slowing down developer velocity in areas where abstraction isn’t necessary. Developers are looking for a combination of flexibility and simplicity depending on the features they are trying to build. Platform teams must continue to simplify the developer experience while strategically exposing lower-level components where necessary. By exposing the right components, they provide developers with greater flexibility and reduce the time needed to integrate new features back into the platform. Imperative IaC for Platform Teams To enable platform teams to build for higher abstraction and flexibility, offload the complexity from CI/CD pipelines, which are designed for orchestration rather than abstraction. Currently, platform teams typically use either configuration-driven systems or declarative IaC, but both approaches have limitations when it comes to creating sophisticated, dynamic deployments and abstraction. The gap in tooling continues to be a challenge, especially for data and AI platform engineering. While there's a proliferation of data product vendors, few focus on simplifying multi-platform deployments. Instead, there's a surge of governance, lineage, and cataloging products to compensate for the lack of integrations. These solutions aim to solve the complexity that arises from using multiple platforms without addressing the core issue of creating streamlined and flexible deployments across data and AI products and platforms. While infrastructure development tools and frameworks are a highly competitive space, there's still room for tools that can provide the advantage of imperative programming for complex logic, while preserving the audit trails, and version control of declarative IaC for governance and compliance. Summary The value of platform engineering is often less tangible than customer-facing product features. While platform improvements may not directly contribute to top-line growth, they are one of the most effective ways to manage cloud costs, which are often one of the biggest drivers of bottom-line costs. Choosing the right toolings and abstraction levels can not only improve developer experience and productivity but also strengthen security, governance, and compliance across the organization. Infrastructure as code (IaC) has become increasingly challenging as organization and usage scale. Inconsistent configurations, security gaps, and increasing costs remain to be hard to manage without the right level of abstraction, but what's the right level of abstraction? Infrastructure as code (IaC) From DevOps to Infrastructure as Code DevOps started with engineers clicking around the AWS console and writing scripts to automate tasks like creating EC2 instances and setting up VPCs, but as usage grew, Infrastructure as Code (IaC) tools such as Terraform and CloudFormation emerged. Instead of writing shell scripts, infrastructure engineers can write declarative code to define how the cloud resources should be deployed. This is a great step up in terms of abstraction from the one-off automation scripts. Terraform CloudFormation On one hand, the declarative nature of the IaC tools makes it easy enough for most developers to write (you can think of it as configuration instead of code); on the other, it's not abstracted enough to give platform engineers the ability to write modularized code for developers to create infrastructure across the organization. Take resource tagging for example, for both cost tracking and governance purposes, it's a very common use case to want to tag cloud resources by application, team, business unit, cost center, etc. With Terraform, application engineers need to manually specify these tags for each resource. Even if platform engineering teams create the relevant terraform "tag" module to enforce common fields, application engineers still need to provide the tag values every time. The declarative nature of IaC makes it hard for platform engineering teams to automatically populate these tag values from a central registry of applications and teams. This becomes especially problematic as the number of resources and size of the organization grows. This led to the next question: how can we create a higher level of abstraction on top of IaC that addresses these enterprise-scale challenges? Creating Platform Abstraction for Scale Creating the right level of platform abstraction enables large organizations to manage infrastructure at scale more effectively and increase developer velocity. Platform engineering teams often use a mix of strategies, depending on use cases, resources, and expertise within the organization. 1. Terraform Modules or Templates Creating reusable terraform modules or templates is a common and easy way to create higher-level abstractions. While flexible, it's a lower-level abstraction, and making organization-wide changes can be time-consuming as they require review and testing by individual application teams. For organizations lacking resources and expertise, vendors like Resourcely create templated terraform code with enforced practice and security. Resourcely 2. Imperative IaC With declarative IaC, integrating external processes into infrastructure deployment often requires workarounds. Platform teams frequently combine declarative IaC with one-off scripts or API calls in their CI/CD pipelines. Tools like AWS CDK (Cloud Development Kit) , CDK for Terraform , and Pulumi allow teams to generate IaC using imperative programming languages, enabling platform engineers to create infrastructure code alongside dynamic processes. AWS CDK (Cloud Development Kit) CDK for Terraform Pulumi While this provides greater flexibility and abstraction, it poses a higher adoption barrier for application teams less familiar with these "CDK" frameworks. 3. Infrastructure and Platform as a Service At the highest abstraction level, platforms like Databricks , Confluent , and Elastic wrap entire infrastructure deployments, exposing only relevant configurations to developers. This allows the provisioning of complex, distributed systems with minimal effort. Kubernetes similarly abstracts compute resource management, enabling developers to create deployments using high-level configurations. Databricks Confluent Elastic Kubernetes While the high-level abstraction allows platform teams to standardize provisioning, governance, and security at scale for supported use cases, it comes at the cost of reduced flexibility and potential vendor lock-in. New or custom use cases can require much longer implementation cycles. Platform Abstraction: Balancing Flexibility and Control Defining the "right" level of abstraction is never easy. Too little abstraction can lead to frequent manual updates and migrations, inconsistent standards, and security and governance implementations. However, too much abstraction can force teams to adapt their use cases to fit the platform's limitations, slowing down developer velocity. The ideal abstraction varies case by case. For real-time fraud detection or experimental features, teams often need flexibility. Templated terraform might provide sufficient guardrails while enabling developers to easily create custom deployments. In contrast, common use cases like data streaming involving Kafka, connectors, and multi-platform integrations typically require more abstraction. Platform teams often create end-to-end deployments, exposing only necessary configs. However, data pipelines can be nuanced, and platform teams often expose lower-level components, allowing application teams to create custom implementations easily and bring back these features to the platform as they mature. For less complicated scenarios like application deployment, most teams only need to specify a Docker image, CPU, and memory requirements. Platform teams can wrap up the CI/CD process without exposing lower-level components. Knowing the different implementation strategies and levels of abstraction for various scenarios, how can we build platforms that scale? Future of Platform Engineering Data and AI Platform as a Service Kubernetes has simplified infrastructure and application deployment, but data platforms have become increasingly complex. The proliferation of specialized data tools, compounded by AI and ML solutions, has made standardization, integration, security, and governance much harder to manage than infrastructure or application deployment. Vendors like Confluent , Snowflake , Databricks , and dbt are improving the developer experience with more automation and integrations, but they often operate independently. This fragmentation makes standardizing multi-directional integrations across identity and access management, data governance, security, and cost control even more challenging. Developing a standardized, secure, and scalable solution for multi-platform environments is now a fast-evolving area for platform engineering teams. Confluent Snowflake Databricks dbt Higher Abstraction for Standardization, Security, and Governance While application developers can use low-level terraform to create infrastructure on their own, this approach doesn’t scale at the enterprise level. Standardization, security, governance, and cost efficiency aren't just “nice to have” but "required" for large organizations. Many companies undertake multi-year, large-scale migrations due to a lack of abstraction where it’s needed, meanwhile slowing down developer velocity in areas where abstraction isn’t necessary. Developers are looking for a combination of flexibility and simplicity depending on the features they are trying to build. Platform teams must continue to simplify the developer experience while strategically exposing lower-level components where necessary. By exposing the right components, they provide developers with greater flexibility and reduce the time needed to integrate new features back into the platform. Imperative IaC for Platform Teams To enable platform teams to build for higher abstraction and flexibility, offload the complexity from CI/CD pipelines, which are designed for orchestration rather than abstraction. Currently, platform teams typically use either configuration-driven systems or declarative IaC, but both approaches have limitations when it comes to creating sophisticated, dynamic deployments and abstraction. The gap in tooling continues to be a challenge, especially for data and AI platform engineering. While there's a proliferation of data product vendors, few focus on simplifying multi-platform deployments. Instead, there's a surge of governance, lineage, and cataloging products to compensate for the lack of integrations. These solutions aim to solve the complexity that arises from using multiple platforms without addressing the core issue of creating streamlined and flexible deployments across data and AI products and platforms. While infrastructure development tools and frameworks are a highly competitive space, there's still room for tools that can provide the advantage of imperative programming for complex logic, while preserving the audit trails, and version control of declarative IaC for governance and compliance. Summary The value of platform engineering is often less tangible than customer-facing product features. While platform improvements may not directly contribute to top-line growth, they are one of the most effective ways to manage cloud costs, which are often one of the biggest drivers of bottom-line costs. Choosing the right toolings and abstraction levels can not only improve developer experience and productivity but also strengthen security, governance, and compliance across the organization.

Stories, News & Mentions about ABSTRACT

Abstract’s Stories on HackerNoon