Cura

cura

Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Think-and-Execute 2 Think-and-Execute 3 Experimental Setup 3 Experimental Setup 4 Results 4 Results 5 Analysis 5 Analysis 6 Related Work 6 Related Work 7 Limitations and Discussion 7 Limitations and Discussion 8 Conclusion and References 8 Conclusion and References A Experimental Details A Experimental Details B Details of Think-and-Execute B Details of Think-and-Execute C Prompts Used in Our Experiments C Prompts Used in Our Experiments D Human-written Pseudocode Prompts D Human-written Pseudocode Prompts E Generated Analyses E Generated Analyses F Generated Pseudocode Prompts F Generated Pseudocode Prompts G Qualitative Analysis G Qualitative Analysis 3 Experimental Setup 3.1 Datasets We curate seven algorithmic reasoning tasks from Big-Bench Hard (Suzgun et al., 2022), including: dyck languages; geometric shapes; navigate; reasoning about colored objects; temporal sequence;tracking shuffled objectives; web of lies. These are specifically designed to measure the step-by-step reasoning capability of LLMs. Model performance on evaluated in zero-shot settings, where we do not provide demonstrations in the prompt. We provide detailed explanations in Appendix A.4. 3.2 Baselines We consider the following baselines: (1) Direct prompting: Directly predicting the answer without generating any rationales. (2) Zero-shot CoT (Kojima et al., 2022): A setting where LLMs are evoked to generate the reasoning steps with “Let’s think step by step”, before the answer. (3) Zero-shot PoT (Chen et al., 2023): A setting where an LLM generates an instancespecific Python code that can be executed with a Python interpreter. Then, the execution result is used as the final answer. (4) NL planning: A variation of THINK-AND-EXECUTE, where the task-level instruction is generated in natural language, instead of pseudocode. Direct prompting Zero-shot CoT Zero-shot PoT NL planning 3.3 Models For the Reasoner LM R, we adopt GPT-3.5-Turbo (OpenAI, 2023), which shows strong performance in various reasoning benchmarks and code generation tasks (Zellers et al., 2019; Cobbe et al., 2021; Muennighoff et al., 2024), as well as the 7B and 13B versions of CodeLlama (Roziere et al., 2023), which are trained on both code and natural language corpora and further fine-tuned to follow natural language instructions. As for the Instructor LM I, we choose GPT-3.5-Turbo. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv Authors:
(1) Hyungjoo Chae, Yonsei University;
(2) Yeonghyeon Kim, Yonsei University;
(3) Seungone Kim, KAIST AI;
(4) Kai Tzu-iunn Ong, Yonsei University;
(5) Beong-woo Kwak, Yonsei University;
(6) Moohyeon Kim, Yonsei University;
(7) Seonghwan Kim, Yonsei University;
(8) Taeyoon Kwon, Yonsei University;
(9) Jiwan Chung, Yonsei University;
(10) Youngjae Yu, Yonsei University;
(11) Jinyoung Yeo, Yonsei University. Authors: Authors: (1) Hyungjoo Chae, Yonsei University; (2) Yeonghyeon Kim, Yonsei University; (3) Seungone Kim, KAIST AI; (4) Kai Tzu-iunn Ong, Yonsei University; (5) Beong-woo Kwak, Yonsei University; (6) Moohyeon Kim, Yonsei University; (7) Seonghwan Kim, Yonsei University; (8) Taeyoon Kwon, Yonsei University; (9) Jiwan Chung, Yonsei University; (10) Youngjae Yu, Yonsei University; (11) Jinyoung Yeo, Yonsei University.

How We Curated Seven Algorithmic Reasoning Tasks From Big-Bench Hard

Table of Links Abstract and 1 Introduction 2. Related Work 3. Method and 3.1. Architecture 3.2. Loss and 3.3. Implementation Details 4. Data Curation 4.1. Training Dataset 4.2. Evaluation Benchmark 5. Experiments and 5.1. Metrics 5.2. Baselines 5.3. Comparison to SOTA Methods 5.4. Qualitative Results and 5.5. Ablation Study 6. Limitations and Discussion 7. Conclusion and References A. Additional Qualitative Comparison B. Inference on AI-generated Images C. Data Curation Details C. Data Curation Details In this section we describe our data generation procedure for training and for rendering the object scans from OmniObject3D to generate one of our benchmark test sets. C.1. Synthetic Training Dataset Generation Image Rendering. For an arbitrary 3D mesh asset, our Blender-based rendering pipeline first loads it into a scene and normalizes it to fit inside a unit cube. Our scene consists of a large rectangular bowl with a flat bottom, a common scene setup that 3D artists use for rendering to allow for realistic shading, and 4 point light sources and one area light source. We randomly place cameras around the object with 30mm to 70mm focal length for a 35mm sensor size equivalent. We randomly vary the distance, elevation (from 5 to 65 degrees), the LookAt point of the camera and generate images of 600 × 600 resolution (see Fig. 11). This variation in object/camera geometry allows capturing the variability of projective geometry in real world scenarios, coming from different capture devices and camera poses. This is in contrast with prior work that uses fixed intrinsics, fixed distance, and LookAt pointed at the center of the object. In addition to RGB images, we extract segmentation masks, depth maps, intrinsics, extrinsics and object pose. We center crop the objects, mask out the background, resize images to 224 × 224 and process the additional annotations to account for the crop, segmentation and resize. C.2. Generating the OmniObject3D Testing Set The original videos released by the OmniObject3D dataset have noisy foreground masks and are mostly taken indoor on a tabletop. To improve the lighting variability and ensure accurate segmentations, we follow the rendering procedure described in the previous section to generate testing data. Different from our training set generation, we use HDRI environment maps to generate scene lighting, which results in high lighting quality and diversity (see Fig. 12). This paper is available on arxiv under CC BY 4.0 DEED license. [6] https://github.com/autonomousvision/occupancy_networks [7] https://github.com/laughtervv/DISN Authors:
(1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work;
(2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work;
(3) Anh Thai, Georgia Institute of Technology;
(4) Varun Jampani, Stability AI;
(5) James M. Rehg, University of Illinois at Urbana-Champaign. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Related Work 2. Related Work 3. Method and 3.1. Architecture 3. Method and 3.1. Architecture 3.2. Loss and 3.3. Implementation Details 3.2. Loss and 3.3. Implementation Details 4. Data Curation 4.1. Training Dataset 4.1. Training Dataset 4.2. Evaluation Benchmark 4.2. Evaluation Benchmark 5. Experiments and 5.1. Metrics 5. Experiments and 5.1. Metrics 5.2. Baselines 5.2. Baselines 5.3. Comparison to SOTA Methods 5.3. Comparison to SOTA Methods 5.4. Qualitative Results and 5.5. Ablation Study 5.4. Qualitative Results and 5.5. Ablation Study 6. Limitations and Discussion 6. Limitations and Discussion 7. Conclusion and References 7. Conclusion and References A. Additional Qualitative Comparison A. Additional Qualitative Comparison B. Inference on AI-generated Images B. Inference on AI-generated Images C. Data Curation Details C. Data Curation Details C. Data Curation Details In this section we describe our data generation procedure for training and for rendering the object scans from OmniObject3D to generate one of our benchmark test sets. C.1. Synthetic Training Dataset Generation Image Rendering. For an arbitrary 3D mesh asset, our Blender-based rendering pipeline first loads it into a scene and normalizes it to fit inside a unit cube. Our scene consists of a large rectangular bowl with a flat bottom, a common scene setup that 3D artists use for rendering to allow for realistic shading, and 4 point light sources and one area light source. We randomly place cameras around the object with 30mm to 70mm focal length for a 35mm sensor size equivalent. We randomly vary the distance, elevation (from 5 to 65 degrees), the LookAt point of the camera and generate images of 600 × 600 resolution (see Fig. 11). This variation in object/camera geometry allows capturing the variability of projective geometry in real world scenarios, coming from different capture devices and camera poses. This is in contrast with prior work that uses fixed intrinsics, fixed distance, and LookAt pointed at the center of the object. Image Rendering. In addition to RGB images, we extract segmentation masks, depth maps, intrinsics, extrinsics and object pose. We center crop the objects, mask out the background, resize images to 224 × 224 and process the additional annotations to account for the crop, segmentation and resize. C.2. Generating the OmniObject3D Testing Set The original videos released by the OmniObject3D dataset have noisy foreground masks and are mostly taken indoor on a tabletop. To improve the lighting variability and ensure accurate segmentations, we follow the rendering procedure described in the previous section to generate testing data. Different from our training set generation, we use HDRI environment maps to generate scene lighting, which results in high lighting quality and diversity (see Fig. 12). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv [6] https://github.com/autonomousvision/occupancy_networks [7] https://github.com/laughtervv/DISN Authors: (1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work; (2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work; (3) Anh Thai, Georgia Institute of Technology; (4) Varun Jampani, Stability AI; (5) James M. Rehg, University of Illinois at Urbana-Champaign. Authors: Authors: (1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work; (2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work; (3) Anh Thai, Georgia Institute of Technology; (4) Varun Jampani, Stability AI; (5) James M. Rehg, University of Illinois at Urbana-Champaign.

ZeroShape: Data Curation Details - Synthetic Training Dataset Generation and More

Table of Links Abstract and 1 Introduction 2. Related Work 3. Method and 3.1. Architecture 3.2. Loss and 3.3. Implementation Details 4. Data Curation 4.1. Training Dataset 4.2. Evaluation Benchmark 5. Experiments and 5.1. Metrics 5.2. Baselines 5.3. Comparison to SOTA Methods 5.4. Qualitative Results and 5.5. Ablation Study 6. Limitations and Discussion 7. Conclusion and References A. Additional Qualitative Comparison B. Inference on AI-generated Images C. Data Curation Details 4. Data Curation 4.1. Training Dataset We use all the 55 categories of ShapeNetCore.v2 [6] for a total of about 52K meshes, as well as over 1000 categories from the Objaverse-LVIS [9] subset. This subset of Objaverse has been manually filtered by crowd workers to primarily include meshes of objects instead of other assets like scans of large scenes and buildings. After filtering Objaverse-LVIS to remove objects with minimal geometry (e.g. objects consisting of a single plane) this dataset has 42K meshes. Pooling these two data sources gives us a total of over 90K 3D object meshes from over 1000 categories. We use Blender [41] to generate synthetic images from the 3D meshes, and to extract a variety of useful annotations: depth maps, camera intrinsics, and object and camera pose. Because the object distribution of ShapeNet is highly skewed (67% of data is 7 categories), we generate 1 to 20 images per object, scaled inversely from the number of meshes in the category of the object, resulting in a total of 159K images. For Objaverse we generate 25 images per object resulting in 939K images. Our traning set consists of slightly less than 1.1M images. We generate images with varying focal lengths, from 30mm to 70mm for a 35mm image sensor size equivalent. We generate diverse object-camera geometry: rather than the common approach of always pointing the camera at the middle of the object at a fixed distance, we vary the object camera distance and vary the LookAt point of the camera. This allows us to capture a wide range of variability in how 3D shape projects to 2D. We follow the convention to use center cropped and foreground segmented images for training and testing. We provide more details in the supplement. This paper is available on arxiv under CC BY 4.0 DEED license. Authors:
(1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work;
(2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work;
(3) Anh Thai, Georgia Institute of Technology;
(4) Varun Jampani, Stability AI;
(5) James M. Rehg, University of Illinois at Urbana-Champaign. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Related Work 2. Related Work 3. Method and 3.1. Architecture 3. Method and 3.1. Architecture 3.2. Loss and 3.3. Implementation Details 3.2. Loss and 3.3. Implementation Details 4. Data Curation 4.1. Training Dataset 4.1. Training Dataset 4.2. Evaluation Benchmark 4.2. Evaluation Benchmark 5. Experiments and 5.1. Metrics 5. Experiments and 5.1. Metrics 5.2. Baselines 5.2. Baselines 5.3. Comparison to SOTA Methods 5.3. Comparison to SOTA Methods 5.4. Qualitative Results and 5.5. Ablation Study 5.4. Qualitative Results and 5.5. Ablation Study 6. Limitations and Discussion 6. Limitations and Discussion 7. Conclusion and References 7. Conclusion and References A. Additional Qualitative Comparison A. Additional Qualitative Comparison B. Inference on AI-generated Images B. Inference on AI-generated Images C. Data Curation Details C. Data Curation Details 4. Data Curation 4.1. Training Dataset We use all the 55 categories of ShapeNetCore.v2 [6] for a total of about 52K meshes, as well as over 1000 categories from the Objaverse-LVIS [9] subset. This subset of Objaverse has been manually filtered by crowd workers to primarily include meshes of objects instead of other assets like scans of large scenes and buildings. After filtering Objaverse-LVIS to remove objects with minimal geometry (e.g. objects consisting of a single plane) this dataset has 42K meshes. Pooling these two data sources gives us a total of over 90K 3D object meshes from over 1000 categories. We use Blender [41] to generate synthetic images from the 3D meshes, and to extract a variety of useful annotations: depth maps, camera intrinsics, and object and camera pose. Because the object distribution of ShapeNet is highly skewed (67% of data is 7 categories), we generate 1 to 20 images per object, scaled inversely from the number of meshes in the category of the object, resulting in a total of 159K images. For Objaverse we generate 25 images per object resulting in 939K images. Our traning set consists of slightly less than 1.1M images. We generate images with varying focal lengths, from 30mm to 70mm for a 35mm image sensor size equivalent. We generate diverse object-camera geometry: rather than the common approach of always pointing the camera at the middle of the object at a fixed distance, we vary the object camera distance and vary the LookAt point of the camera. This allows us to capture a wide range of variability in how 3D shape projects to 2D. We follow the convention to use center cropped and foreground segmented images for training and testing. We provide more details in the supplement. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv Authors: (1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work; (2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work; (3) Anh Thai, Georgia Institute of Technology; (4) Varun Jampani, Stability AI; (5) James M. Rehg, University of Illinois at Urbana-Champaign. Authors: Authors: (1) Zixuan Huang, University of Illinois at Urbana-Champaign and both authors contributed equally to this work; (2) Stefan Stojanov, Georgia Institute of Technology and both authors contributed equally to this work; (3) Anh Thai, Georgia Institute of Technology; (4) Varun Jampani, Stability AI; (5) James M. Rehg, University of Illinois at Urbana-Champaign.

ZeroShape: Here's How We Did Our Data Curation

Table of Links Abstract and 1. Introduction 2 The Original GPT4All Model 2.1 Data Collection and Curation 2.2 Model Training, 2.3 Model Access and 2.4 Model Evaluation 3 From a Model to an Ecosystem 3.1 GPT4All-J: Repository Growth and the implications of the LLaMA License 3.2 GPT4All-Snoozy: the Emergence of the GPT4All Ecosystem 3.3 The Current State of GPT4All 4 The Future of GPT4All Limitations and References 2 The Original GPT4All Model 2.1 Data Collection and Curation To train the original GPT4All model, we collected roughly one million prompt-response pairs using the GPT-3.5-Turbo OpenAI API between March 20, 2023 and March 26th, 2023. In particular, we gathered GPT3.5-Turbo responses to prompts of three publicly available datasets: the unified chip2 subset of LAION OIG, a random sub-sample of Stackoverflow Questions, and a sub-sample of Bigscience/P3 (Sanh et al., 2021). Following the approach in Stanford Alpaca (Taori et al., 2023), an open source LLaMA variant that came just before GPT4All, we focused substantial effort on dataset curation. The collected dataset was loaded into Atlas (AI, 2023)—a visual interface for exploring and tagging massive unstructured datasets —for data curation. Using AtarXiv:2311.04931v1 [cs.CL] 6 Nov 2023 las, we identified and removed subsets of the data where GPT-3.5-Turbo refused to respond, had malformed output, or produced a very short response. This resulted in the removal of the entire Bigscience/P3 subset of our data, as many P3 prompts induced responses that were simply one word. After curation, we were left with a set of 437,605 prompt-response pairs, which we visualize in Figure 1a. This paper is available on arxiv under CC BY 4.0 DEED license. Authors:
(1) Yuvanesh Anand, Nomic AI, yuvanesh@nomic.ai;
(2) Zach Nussbaum, Nomic AI, zach@nomic.ai;
(3) Adam Treat, Nomic AI, adam@nomic.ai;
(4) Aaron Miller, Nomic AI, aaron@nomic.ai;
(5) Richard Guo, Nomic AI, richard@nomic.ai;
(6) Ben Schmidt, Nomic AI, ben@nomic.ai;
(7) GPT4All Community, Planet Earth;
(8) Brandon Duderstadt, Nomic AI, brandon@nomic.ai with Shared Senior Authorship;
(9) Andriy Mulyar, Nomic AI, andriy@nomic.ai with Shared Senior Authorship. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 The Original GPT4All Model 2 The Original GPT4All Model 2.1 Data Collection and Curation 2.2 Model Training, 2.3 Model Access and 2.4 Model Evaluation 2.2 Model Training, 2.3 Model Access and 2.4 Model Evaluation 3 From a Model to an Ecosystem 3.1 GPT4All-J: Repository Growth and the implications of the LLaMA License 3.1 GPT4All-J: Repository Growth and the implications of the LLaMA License 3.2 GPT4All-Snoozy: the Emergence of the GPT4All Ecosystem 3.2 GPT4All-Snoozy: the Emergence of the GPT4All Ecosystem 3.3 The Current State of GPT4All 3.3 The Current State of GPT4All 4 The Future of GPT4All 4 The Future of GPT4All Limitations and References Limitations and References 2 The Original GPT4All Model 2.1 Data Collection and Curation To train the original GPT4All model, we collected roughly one million prompt-response pairs using the GPT-3.5-Turbo OpenAI API between March 20, 2023 and March 26th, 2023. In particular, we gathered GPT3.5-Turbo responses to prompts of three publicly available datasets: the unified chip2 subset of LAION OIG, a random sub-sample of Stackoverflow Questions, and a sub-sample of Bigscience/P3 (Sanh et al., 2021). Following the approach in Stanford Alpaca (Taori et al., 2023), an open source LLaMA variant that came just before GPT4All, we focused substantial effort on dataset curation. The collected dataset was loaded into Atlas (AI, 2023)—a visual interface for exploring and tagging massive unstructured datasets —for data curation. Using AtarXiv:2311.04931v1 [cs.CL] 6 Nov 2023 las, we identified and removed subsets of the data where GPT-3.5-Turbo refused to respond, had malformed output, or produced a very short response. This resulted in the removal of the entire Bigscience/P3 subset of our data, as many P3 prompts induced responses that were simply one word. After curation, we were left with a set of 437,605 prompt-response pairs, which we visualize in Figure 1a. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv Authors: (1) Yuvanesh Anand, Nomic AI, yuvanesh@nomic.ai; (2) Zach Nussbaum, Nomic AI, zach@nomic.ai; (3) Adam Treat, Nomic AI, adam@nomic.ai; (4) Aaron Miller, Nomic AI, aaron@nomic.ai; (5) Richard Guo, Nomic AI, richard@nomic.ai; (6) Ben Schmidt, Nomic AI, ben@nomic.ai; (7) GPT4All Community, Planet Earth; (8) Brandon Duderstadt, Nomic AI, brandon@nomic.ai with Shared Senior Authorship; (9) Andriy Mulyar, Nomic AI, andriy@nomic.ai with Shared Senior Authorship. Authors: Authors: (1) Yuvanesh Anand, Nomic AI, yuvanesh@nomic.ai; (2) Zach Nussbaum, Nomic AI, zach@nomic.ai; (3) Adam Treat, Nomic AI, adam@nomic.ai; (4) Aaron Miller, Nomic AI, aaron@nomic.ai; (5) Richard Guo, Nomic AI, richard@nomic.ai; (6) Ben Schmidt, Nomic AI, ben@nomic.ai; (7) GPT4All Community, Planet Earth; (8) Brandon Duderstadt, Nomic AI, brandon@nomic.ai with Shared Senior Authorship; (9) Andriy Mulyar, Nomic AI, andriy@nomic.ai with Shared Senior Authorship.

Original GPT4All Model: How We Collected Data and Then Curated It

Learn how to programmatically fetch and embed your liked Bluesky posts using authentication, API calls, and oEmbed endpoints. I've recently made the move over to Bluesky. I can already confirm that there is a vibrant tech community there with tons of interesting, useful, and inspiring content. I'm a happy new user! As a result, I've been wanting to embed my top liked Bluesky posts in my "Dev roundup" monthly newsletter posts. My aim is to provide a curated list of Bluesky posts that is specifically tailored to Software Developers. Luckily, Bluesky's API is completely free to use, allowing programmatic access to all of the content within. This tutorial will walk you through the process of retrieving and embedding liked Bluesky posts using their API, perfect for personal blogs, portfolios, or content aggregation projects. Understanding my Bluesky API Workflow I have built a script that allows me to automatically embed my Bluesky posts in a markdown blog post. I think that any or all of the steps used in this script are valuable for many use-cases. To summarize my workflow for embedding liked posts, we follow these key steps: Create an authenticated session
Retrieve liked post URIs for an "actor"
Use these URIs to fetch oEmbed embed HTML
Clean and format the embed code The Complete Implementation Let's break down each function and its purpose: 1. Creating a Bluesky Session export const createSession = async (): Promise<string | null> => {
  try {
    const response = await fetch(
      "https://bsky.social/xrpc/com.atproto.server.createSession",
      {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          identifier: "your-handle",
          password: "your-password",
        }),
      }
    );
    const responseJson = await response.json();
    return responseJson.accessJwt;
  } catch (e) {
    console.error("Failed to create session: ", e);
    return null;
  }
}; Key Insights: This function authenticates your Bluesky account.

Note: this example hardcodes strings for credentials, but this should be avoided in production use cases.


It returns an accessJwt JWT (JSON Web Token) for subsequent API calls
Uses the createSession endpoint from Bluesky's ATP (Authenticated Transfer Protocol)
Error handling ensures graceful failure if authentication fails 2. Retrieving Liked Post URIs export const getBlueskyLikeUris = async (actor: string, limit: number = 40) => {
  const token = await createSession();
  if (!token) {
    console.error("Failed to get token");
    return;
  }
  
  const response = await fetch(
    "https://bsky.social/xrpc/app.bsky.feed.getActorLikes?actor=${actor}&limit=${limit}",
    {
      method: "GET",
      headers: {
        Authorization: `Bearer ${token}`,
      },
    }
  );
  
  const responseJson = await response.json();
  const uris = responseJson.feed.map((entry: any) => entry.post.uri);

  return uris;
}; Key Insights: Requires an authenticated session token
Uses the getActorLikes endpoint to fetch liked posts
Important -- the endpoint domain should be https://bsky.social, as this is an authenticated request.
Extracts unique URIs for each liked post
Limits to 40 posts (configurable) 3. Converting URIs to Embeddable HTML export const getBlueskyPostEmbedMarkup = async (uri: string) => {
  try {
    const response = await fetch(`https://embed.bsky.app/oembed?url=${uri}`);
    const responseJson = await response.json();
    
    const formattedHTML = prettier.format(responseJson.html, {
      parser: "html",
      plugins: [require("prettier/parser-html")],
      htmlWhitespaceSensitivity: "ignore",
      printWidth: 1000,
    });
    
    return formattedHTML.replace(/<script[\s\S]*?<\/script>/g, "");
  } catch (e) {
    console.error("Failed to get Bluesky post embed markup");
    return null;
  }
}; Key Insights: Uses Bluesky's oEmbed endpoint with post URIs to access a post's embeddable HTML
Optional: Utilizes prettier to format the HTML consistently
Optional: Removes <script> tags for security and clean embedding

The reason for this is that I embed a single Bluesky script for each post containing Bluesky content.


Flexible error handling Putting It All Together: A Complete Example async function embedLikedPosts() {
  try {
    // Get liked post URIs
    const likedPostUris = await getBlueskyLikeUris();
    
    if (!likedPostUris) {
      console.error("No liked posts found");
      return;
    }
    
    // Convert URIs to embed HTML
    const embedPromises = likedPostUris.map(getBlueskyPostEmbedMarkup);
    const embedHtmlArray = await Promise.all(embedPromises);
    
    // Filter out any failed embeds
    const validEmbeds = embedHtmlArray.filter(embed => embed !== null);
    
    // Return the markup for all liked posts
    return `
      ## Some Fave Posts 🦋
      ${validEmbeds.join(`\n\n`)}
    `
  } catch (error) {
    console.error("Error embedding Bluesky posts:", error);
  }
} Potential Enhancements This solution works for me because all that I require is a statically generated monthly blog post. Some improvements could include: Add pagination support for fetching more than 40 liked posts
Implement caching to reduce unnecessary API calls
Create a more robust error handling mechanism
Creating mechanism for refreshing accessJwt token if used in long running processes
Sorting liked posts by popularity (likes) Troubleshooting Tips Verify your Bluesky credentials are correct
Check that the Bearer token is being set correctly on your authenticated requests.
Verify that the endpoint domains you are using are all valid. Conclusion Embedding Bluesky posts provides a dynamic way to showcase your social media interactions. By understanding the API workflow and implementing robust error handling, you can create engaging, personalized, and curated content integrations. Next Steps Experiment with the code
Customize the embed styling
Explore additional Bluesky API endpoints Enjoy and happy tinkering! 🚀 Learn how to programmatically fetch and embed your liked Bluesky posts using authentication, API calls, and oEmbed endpoints. Learn how to programmatically fetch and embed your liked Bluesky posts using authentication, API calls, and oEmbed endpoints. I've recently made the move over to Bluesky . I can already confirm that there is a vibrant tech community there with tons of interesting, useful, and inspiring content. I'm a happy new user! As a result, I've been wanting to embed my top liked Bluesky posts in my "Dev roundup" monthly newsletter posts. My aim is to provide a curated list of Bluesky posts that is specifically tailored to Software Developers. Bluesky Luckily, Bluesky's API is completely free to use, allowing programmatic access to all of the content within. This tutorial will walk you through the process of retrieving and embedding liked Bluesky posts using their API, perfect for personal blogs, portfolios, or content aggregation projects. Bluesky's API Understanding my Bluesky API Workflow I have built a script that allows me to automatically embed my Bluesky posts in a markdown blog post. I think that any or all of the steps used in this script are valuable for many use-cases. To summarize my workflow for embedding liked posts, we follow these key steps: Create an authenticated session Retrieve liked post URIs for an "actor" Use these URIs to fetch oEmbed embed HTML Clean and format the embed code Create an authenticated session Retrieve liked post URIs for an "actor" Use these URIs to fetch oEmbed embed HTML oEmbed embed HTML Clean and format the embed code The Complete Implementation Let's break down each function and its purpose: 1. Creating a Bluesky Session export const createSession = async (): Promise<string | null> => {
  try {
    const response = await fetch(
      "https://bsky.social/xrpc/com.atproto.server.createSession",
      {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          identifier: "your-handle",
          password: "your-password",
        }),
      }
    );
    const responseJson = await response.json();
    return responseJson.accessJwt;
  } catch (e) {
    console.error("Failed to create session: ", e);
    return null;
  }
}; export const createSession = async (): Promise<string | null> => {
  try {
    const response = await fetch(
      "https://bsky.social/xrpc/com.atproto.server.createSession",
      {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          identifier: "your-handle",
          password: "your-password",
        }),
      }
    );
    const responseJson = await response.json();
    return responseJson.accessJwt;
  } catch (e) {
    console.error("Failed to create session: ", e);
    return null;
  }
}; Key Insights: Key Insights: This function authenticates your Bluesky account.

Note: this example hardcodes strings for credentials, but this should be avoided in production use cases. It returns an accessJwt JWT (JSON Web Token) for subsequent API calls Uses the createSession endpoint from Bluesky's ATP (Authenticated Transfer Protocol) Error handling ensures graceful failure if authentication fails This function authenticates your Bluesky account. Note: this example hardcodes strings for credentials, but this should be avoided in production use cases. Note: this example hardcodes strings for credentials, but this should be avoided in production use cases. Note: this example hardcodes strings for credentials, but this should be avoided in production use cases. It returns an accessJwt JWT (JSON Web Token) for subsequent API calls accessJwt Uses the createSession endpoint from Bluesky's ATP (Authenticated Transfer Protocol) createSession Error handling ensures graceful failure if authentication fails 2. Retrieving Liked Post URIs export const getBlueskyLikeUris = async (actor: string, limit: number = 40) => {
  const token = await createSession();
  if (!token) {
    console.error("Failed to get token");
    return;
  }
  
  const response = await fetch(
    "https://bsky.social/xrpc/app.bsky.feed.getActorLikes?actor=${actor}&limit=${limit}",
    {
      method: "GET",
      headers: {
        Authorization: `Bearer ${token}`,
      },
    }
  );
  
  const responseJson = await response.json();
  const uris = responseJson.feed.map((entry: any) => entry.post.uri);

  return uris;
}; export const getBlueskyLikeUris = async (actor: string, limit: number = 40) => {
  const token = await createSession();
  if (!token) {
    console.error("Failed to get token");
    return;
  }
  
  const response = await fetch(
    "https://bsky.social/xrpc/app.bsky.feed.getActorLikes?actor=${actor}&limit=${limit}",
    {
      method: "GET",
      headers: {
        Authorization: `Bearer ${token}`,
      },
    }
  );
  
  const responseJson = await response.json();
  const uris = responseJson.feed.map((entry: any) => entry.post.uri);

  return uris;
}; Key Insights: Key Insights: Requires an authenticated session token Uses the getActorLikes endpoint to fetch liked posts Important -- the endpoint domain should be https://bsky.social, as this is an authenticated request. Extracts unique URIs for each liked post Limits to 40 posts (configurable) Requires an authenticated session token Uses the getActorLikes endpoint to fetch liked posts the getActorLikes Important -- the endpoint domain should be https://bsky.social, as this is an authenticated request. an authenticated request. Extracts unique URIs for each liked post Limits to 40 posts (configurable) 3. Converting URIs to Embeddable HTML export const getBlueskyPostEmbedMarkup = async (uri: string) => {
  try {
    const response = await fetch(`https://embed.bsky.app/oembed?url=${uri}`);
    const responseJson = await response.json();
    
    const formattedHTML = prettier.format(responseJson.html, {
      parser: "html",
      plugins: [require("prettier/parser-html")],
      htmlWhitespaceSensitivity: "ignore",
      printWidth: 1000,
    });
    
    return formattedHTML.replace(/<script[\s\S]*?<\/script>/g, "");
  } catch (e) {
    console.error("Failed to get Bluesky post embed markup");
    return null;
  }
}; export const getBlueskyPostEmbedMarkup = async (uri: string) => {
  try {
    const response = await fetch(`https://embed.bsky.app/oembed?url=${uri}`);
    const responseJson = await response.json();
    
    const formattedHTML = prettier.format(responseJson.html, {
      parser: "html",
      plugins: [require("prettier/parser-html")],
      htmlWhitespaceSensitivity: "ignore",
      printWidth: 1000,
    });
    
    return formattedHTML.replace(/<script[\s\S]*?<\/script>/g, "");
  } catch (e) {
    console.error("Failed to get Bluesky post embed markup");
    return null;
  }
}; Key Insights: Key Insights: Uses Bluesky's oEmbed endpoint with post URIs to access a post's embeddable HTML Optional: Utilizes prettier to format the HTML consistently Optional: Removes <script> tags for security and clean embedding

The reason for this is that I embed a single Bluesky script for each post containing Bluesky content. Flexible error handling Uses Bluesky's oEmbed endpoint with post URIs to access a post's embeddable HTML Bluesky's oEmbed endpoint Optional: Utilizes prettier to format the HTML consistently prettier Optional: Removes <script> tags for security and clean embedding The reason for this is that I embed a single Bluesky script for each post containing Bluesky content. <script> The reason for this is that I embed a single Bluesky script for each post containing Bluesky content. The reason for this is that I embed a single Bluesky script for each post containing Bluesky content. Flexible error handling Putting It All Together: A Complete Example async function embedLikedPosts() {
  try {
    // Get liked post URIs
    const likedPostUris = await getBlueskyLikeUris();
    
    if (!likedPostUris) {
      console.error("No liked posts found");
      return;
    }
    
    // Convert URIs to embed HTML
    const embedPromises = likedPostUris.map(getBlueskyPostEmbedMarkup);
    const embedHtmlArray = await Promise.all(embedPromises);
    
    // Filter out any failed embeds
    const validEmbeds = embedHtmlArray.filter(embed => embed !== null);
    
    // Return the markup for all liked posts
    return `
      ## Some Fave Posts 🦋
      ${validEmbeds.join(`\n\n`)}
    `
  } catch (error) {
    console.error("Error embedding Bluesky posts:", error);
  }
} async function embedLikedPosts() {
  try {
    // Get liked post URIs
    const likedPostUris = await getBlueskyLikeUris();
    
    if (!likedPostUris) {
      console.error("No liked posts found");
      return;
    }
    
    // Convert URIs to embed HTML
    const embedPromises = likedPostUris.map(getBlueskyPostEmbedMarkup);
    const embedHtmlArray = await Promise.all(embedPromises);
    
    // Filter out any failed embeds
    const validEmbeds = embedHtmlArray.filter(embed => embed !== null);
    
    // Return the markup for all liked posts
    return `
      ## Some Fave Posts 🦋
      ${validEmbeds.join(`\n\n`)}
    `
  } catch (error) {
    console.error("Error embedding Bluesky posts:", error);
  }
} Potential Enhancements This solution works for me because all that I require is a statically generated monthly blog post. Some improvements could include: Add pagination support for fetching more than 40 liked posts Implement caching to reduce unnecessary API calls Create a more robust error handling mechanism Creating mechanism for refreshing accessJwt token if used in long running processes Sorting liked posts by popularity (likes) Add pagination support for fetching more than 40 liked posts Implement caching to reduce unnecessary API calls Create a more robust error handling mechanism Creating mechanism for refreshing accessJwt token if used in long running processes accessJwt Sorting liked posts by popularity (likes) Troubleshooting Tips Verify your Bluesky credentials are correct Check that the Bearer token is being set correctly on your authenticated requests. Verify that the endpoint domains you are using are all valid. Verify your Bluesky credentials are correct Check that the Bearer token is being set correctly on your authenticated requests. Verify that the endpoint domains you are using are all valid. Conclusion Embedding Bluesky posts provides a dynamic way to showcase your social media interactions. By understanding the API workflow and implementing robust error handling, you can create engaging, personalized, and curated content integrations. Next Steps Experiment with the code Customize the embed styling Explore additional Bluesky API endpoints Experiment with the code Customize the embed styling Explore additional Bluesky API endpoints Enjoy and happy tinkering! 🚀

Bluesky's Free API Helps Me Curate My Top Posts Like Never Before

Bluesky ukan API Gratis ukax Nayan Jach’a Posts ukanakar Curar ukham yanapt’itu, kunjamatix Janipuniw Nayrjax ukhama

La API gratuita de Bluesky me ayuda a organizar mis publicaciones más destacadas como nunca antes

API رایگان Bluesky به من کمک می‌کند تا پست‌های برتر خود را مانند قبل تنظیم کنم

A Bluesky ingyenes API-ja úgy segít összeválogatni a legnépszerűbb bejegyzéseimet, mint még soha

API Gratis Bluesky Membantu Saya Menyusun Postingan Terbaik Saya Seperti Sebelumnya

Bluesky の無料 API は、これまでにないほど私の人気投稿をキュレートするのに役立ちます

API ya ofele ya Bluesky Esalisaka Ngai Na Curer Ba Posts Na Ngai Ya Likolo Neti Jamais Avant

Bluesky को नि: शुल्क API ले मलाई मेरो शीर्ष पोष्टहरू पहिले कहिल्यै क्युरेट गर्न मद्दत गर्दछ

Bezpłatne API Bluesky pomaga mi w jak nigdy dotąd selekcjonowaniu moich najlepszych postów

د بلوسکي وړیا API زما سره زما د غوره پوسټونو په ترتیب کولو کې مرسته کوي لکه مخکې هیڅکله

Bluesky's API Bilaashka ah waxay iga caawisaa inaan hagaajiyo qoraaladayda ugu sarreeya si aan waligeed horay loo arag

Бесплатни АПИ Блуески-а ми помаже да бирам моје најпопуларније постове као никада раније

LLMs For Curating Your Social Media Feeds? Yes Please!

Sosial Media Lentləriniz üçün LLM-lər? Bəli Zəhmət olmasa!

আপনার সোশ্যাল মিডিয়া ফিড কিউরেট করার জন্য এলএলএম? হ্যাঁ দয়া করে!

¿Másteres de posgrado para gestionar tus feeds de redes sociales? ¡Sí, por favor!

Zure sare sozialen jarioak kudeatzeko LLMak? Bai Mesedez!

LLM-ek a közösségi média hírcsatornáinak gondozásához? Igen Kérem!

ソーシャル メディア フィードをキュレートするための LLM が必要ですか? はい、ぜひ!

LLM jūsu sociālo mediju plūsmu pārvaldīšanai? Jā Lūdzu!

LLM за средување на вашите доводи на социјалните медиуми? Да Ве молам!

तपाईंको सोशल मिडिया फिडहरू क्युरेट गर्नको लागि LLMs? हो कृपया!

LLM za kuratorowanie Twoich kanałów mediów społecznościowych? Tak, proszę!

உங்கள் சமூக ஊடக ஊட்டங்களைச் சரிசெய்வதற்கான LLMகள்? ஆம் ப்ளீஸ்!

法学硕士可以帮你策划社交媒体信息吗？可以！

Podcast content is growing each year, with over 4.5 million active podcasts globally — and nearly 30 million episodes published in 2023 alone — the challenge of discovering relevant content has become increasingly complex. Podyssey emerged as a solution to this challenge, with its AI-driven approach to podcast curation and personalization. Ahmad Saleem, co-founder and CEO of Podyssey aims to set new standards for how listeners find and engage with relevant podcast content. "In the 1980s and 90s, the antidote to a mid-life crisis was starting a suburban band. Now it is starting a podcast.” Historically, the only way to find relevant podcast content was by manually searching through titles of shows and episode descriptions, relying heavily on ratings, reviews, and basic categorization. But these traditional discovery methods have proven insufficient as the volume of content continues to increase. Listeners often find themselves overwhelmed by choices or stuck in content bubbles, missing out on potentially valuable podcasts that fall outside of their usual listening patterns. “As a listener, the task of shifting through this new content is onerous and daunting. In fact, just finding that content you already like is becoming more and more daunting,” comments Saleem. “This is largely because there is more content to sift through; but also because the method for doing so is increasingly outdated.” Podyssey's approach puts curated content discovery at the forefront. At its heart lies a powerful and unique search engine that leverages cutting-edge technology to understand the actual discussions within podcasts. This isn't just an incremental improvement – it's a complete paradigm shift in how listeners find and consume podcast content. "Traditional platforms have to rely on the genre or content classifications applied by the podcaster to their content," explains Saleem. “This approach might have worked in simpler times, but today's podcast landscape defies such basic categorization. Take Joe Rogan's podcast, for instance – trying to pin it down to a single genre is like trying to fit a square peg in a round hole. The reality is that podcast content has become more genre-fluid than ever before, particularly in long-form formats.” The technical innovation behind Podyssey is substantial. The platform employs a sophisticated combination of audio recognition algorithms and natural language processing, utilizing bi-encoders and cross-encoders to analyze user queries. This goes beyond simple keyword matching, allowing the system to understand the intent behind searches and deliver more relevant results, even without exact matches. "There is nothing else like Podyssey out there," according to Saleem. "We are the only podcast platform that allows listeners to curate their listening based on the content being discussed." This means listeners can finally break free from the constraints of genre-based discovery. Podyssey goes beyond search capabilities. By studying listener behavior – from search patterns to engagement rates with specific clips – the platform creates highly detailed user profiles that enable complete personalization. Instead of sitting through a three-hour episode for ten minutes of relevant content, users can enjoy sixty minutes of carefully curated material drawn from various sources, all focused on their specific interests. The company’s journey is about more than just improving podcast discovery – it's about reimagining the entire podcast ecosystem. While traditional platforms have remained relatively unchanged since the industry's inception, Podyssey represents a fundamental shift in how both creators and listeners engage with podcast content. By removing artificial boundaries and putting content relationships at the center of the experience, Podyssey isn't just participating in the podcast revolution – it's leading it. The message is clear: the future of podcast discovery has arrived, and it's time for listeners to break free from the constraints of traditional platforms. Podyssey stands ready to ensure that every listener can find exactly what they're looking for – and discover content they never knew they needed. Podcast content is growing each year, with over 4.5 million active podcasts globally — and nearly 30 million episodes published in 2023 alone — the challenge of discovering relevant content has become increasingly complex. Podyssey emerged as a solution to this challenge, with its AI-driven approach to podcast curation and personalization. Ahmad Saleem, co-founder and CEO of Podyssey aims to set new standards for how listeners find and engage with relevant podcast content. Podyssey "In the 1980s and 90s, the antidote to a mid-life crisis was starting a suburban band. Now it is starting a podcast.” "In the 1980s and 90s, the antidote to a mid-life crisis was starting a suburban band. Now it is starting a podcast.” Historically, the only way to find relevant podcast content was by manually searching through titles of shows and episode descriptions, relying heavily on ratings, reviews, and basic categorization. But these traditional discovery methods have proven insufficient as the volume of content continues to increase. Listeners often find themselves overwhelmed by choices or stuck in content bubbles, missing out on potentially valuable podcasts that fall outside of their usual listening patterns. “As a listener, the task of shifting through this new content is onerous and daunting. In fact, just finding that content you already like is becoming more and more daunting,” comments Saleem. “This is largely because there is more content to sift through; but also because the method for doing so is increasingly outdated.” Podyssey's approach puts curated content discovery at the forefront. At its heart lies a powerful and unique search engine that leverages cutting-edge technology to understand the actual discussions within podcasts. This isn't just an incremental improvement – it's a complete paradigm shift in how listeners find and consume podcast content. "Traditional platforms have to rely on the genre or content classifications applied by the podcaster to their content," explains Saleem. “This approach might have worked in simpler times, but today's podcast landscape defies such basic categorization. Take Joe Rogan's podcast, for instance – trying to pin it down to a single genre is like trying to fit a square peg in a round hole. The reality is that podcast content has become more genre-fluid than ever before, particularly in long-form formats.” The technical innovation behind Podyssey is substantial. The platform employs a sophisticated combination of audio recognition algorithms and natural language processing, utilizing bi-encoders and cross-encoders to analyze user queries. This goes beyond simple keyword matching, allowing the system to understand the intent behind searches and deliver more relevant results, even without exact matches. "There is nothing else like Podyssey out there," according to Saleem. "We are the only podcast platform that allows listeners to curate their listening based on the content being discussed." This means listeners can finally break free from the constraints of genre-based discovery. Podyssey goes beyond search capabilities. By studying listener behavior – from search patterns to engagement rates with specific clips – the platform creates highly detailed user profiles that enable complete personalization. Instead of sitting through a three-hour episode for ten minutes of relevant content, users can enjoy sixty minutes of carefully curated material drawn from various sources, all focused on their specific interests. The company’s journey is about more than just improving podcast discovery – it's about reimagining the entire podcast ecosystem. While traditional platforms have remained relatively unchanged since the industry's inception, Podyssey represents a fundamental shift in how both creators and listeners engage with podcast content. By removing artificial boundaries and putting content relationships at the center of the experience, Podyssey isn't just participating in the podcast revolution – it's leading it. The message is clear: the future of podcast discovery has arrived, and it's time for listeners to break free from the constraints of traditional platforms. Podyssey stands ready to ensure that every listener can find exactly what they're looking for – and discover content they never knew they needed.

The Future of Podcast Curation: Podyssey's AI-Driven Approach to Content Discovery

Orchestrating Airflow DAGs with GitHub Actions - A Lightweight Approach to Data Curation Across Spa

Authors:
(1) Tony Lee, Stanford with Equal contribution;
(2) Michihiro Yasunaga, Stanford with Equal contribution;
(3) Chenlin Meng, Stanford with Equal contribution;
(4) Yifan Mai, Stanford;
(5) Joon Sung Park, Stanford;
(6) Agrim Gupta, Stanford;
(7) Yunzhi Zhang, Stanford;
(8) Deepak Narayanan, Microsoft;
(9) Hannah Benita Teufel, Aleph Alpha;
(10) Marco Bellagente, Aleph Alpha;
(11) Minguk Kang, POSTECH;
(12) Taesung Park, Adobe;
(13) Jure Leskovec, Stanford;
(14) Jun-Yan Zhu, CMU;
(15) Li Fei-Fei, Stanford;
(16) Jiajun Wu, Stanford;
(17) Stefano Ermon, Stanford;
(18) Percy Liang, Stanford. Table of Links Abstract and 1 Introduction 2 Core framework 3 Aspects 4 Scenarios 5 Metrics 6 Models 7 Experiments and results 8 Related work 9 Conclusion 10 Limitations Author contributions, Acknowledgments and References A Datasheet B Scenario details C Metric details D Model details E Human evaluation procedure 4 Scenarios To evaluate the 12 aspects (§3), we curate diverse and practical scenarios. Table 2 presents an overview of all the scenarios and their descriptions. Each scenario is a set of textual inputs and can be used to evaluate certain aspects. For instance, the “MS-COCO” scenario can be used to assess the alignment, quality, and efficiency aspects, and the “Inappropriate Image Prompts (I2P)” scenario [8] can be used to assess the toxicity aspect. Some scenarios may include sub-scenarios, indicating the sub-level categories or variations within them, such as “Hate” and “Violence” within I2P. We curate these scenarios by leveraging existing datasets and creating new prompts ourselves. In total, we have 62 scenarios, including the sub-scenarios. Notably, we create new scenarios (indicated with “New” in Table 2) for aspects that were previously underexplored and lacked dedicated datasets. These aspects include originality, aesthetics, bias, and fairness. For example, to evaluate originality, we develop scenarios to test the artistic creativity of these models with textual inputs to generate landing pages, logos, and magazine covers. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Tony Lee, Stanford with Equal contribution; (2) Michihiro Yasunaga, Stanford with Equal contribution; (3) Chenlin Meng, Stanford with Equal contribution; (4) Yifan Mai, Stanford; (5) Joon Sung Park, Stanford; (6) Agrim Gupta, Stanford; (7) Yunzhi Zhang, Stanford; (8) Deepak Narayanan, Microsoft; (9) Hannah Benita Teufel, Aleph Alpha; (10) Marco Bellagente, Aleph Alpha; (11) Minguk Kang, POSTECH; (12) Taesung Park, Adobe; (13) Jure Leskovec, Stanford; (14) Jun-Yan Zhu, CMU; (15) Li Fei-Fei, Stanford; (16) Jiajun Wu, Stanford; (17) Stefano Ermon, Stanford; (18) Percy Liang, Stanford. Authors: Authors: (1) Tony Lee, Stanford with Equal contribution; (2) Michihiro Yasunaga, Stanford with Equal contribution; (3) Chenlin Meng, Stanford with Equal contribution; (4) Yifan Mai, Stanford; (5) Joon Sung Park, Stanford; (6) Agrim Gupta, Stanford; (7) Yunzhi Zhang, Stanford; (8) Deepak Narayanan, Microsoft; (9) Hannah Benita Teufel, Aleph Alpha; (10) Marco Bellagente, Aleph Alpha; (11) Minguk Kang, POSTECH; (12) Taesung Park, Adobe; (13) Jure Leskovec, Stanford; (14) Jun-Yan Zhu, CMU; (15) Li Fei-Fei, Stanford; (16) Jiajun Wu, Stanford; (17) Stefano Ermon, Stanford; (18) Percy Liang, Stanford. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Core framework 2 Core framework 3 Aspects 3 Aspects 4 Scenarios 4 Scenarios 5 Metrics 5 Metrics 6 Models 6 Models 7 Experiments and results 7 Experiments and results 8 Related work 8 Related work 9 Conclusion 9 Conclusion 10 Limitations 10 Limitations Author contributions, Acknowledgments and References Author contributions, Acknowledgments and References A Datasheet A Datasheet B Scenario details B Scenario details C Metric details C Metric details D Model details D Model details E Human evaluation procedure E Human evaluation procedure 4 Scenarios To evaluate the 12 aspects (§3), we curate diverse and practical scenarios. Table 2 presents an overview of all the scenarios and their descriptions. Each scenario is a set of textual inputs and can be used to evaluate certain aspects. For instance, the “MS-COCO” scenario can be used to assess the alignment, quality, and efficiency aspects, and the “Inappropriate Image Prompts (I2P)” scenario [8] can be used to assess the toxicity aspect. Some scenarios may include sub-scenarios, indicating the sub-level categories or variations within them, such as “Hate” and “Violence” within I2P. We curate these scenarios by leveraging existing datasets and creating new prompts ourselves. In total, we have 62 scenarios, including the sub-scenarios. Notably, we create new scenarios (indicated with “ New ” in Table 2) for aspects that were previously underexplored and lacked dedicated datasets. These aspects include originality, aesthetics, bias, and fairness. For example, to evaluate originality, we develop scenarios to test the artistic creativity of these models with textual inputs to generate landing pages, logos, and magazine covers. New This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Curating 62 Practical Scenarios to Test AI Text-to-Image Models

Nowadays, we are constantly bombarded with such a high volume of information from various sources. This massive influx of data, coupled with the development of content generation from AI, undoubtedly gives us greater access to knowledge than we’ve ever had before. However, it also brings in the problem of information overload. Think of a typical working day for yourself or a colleague. You might open your inbox, which is absolutely stacked with threads, internal bulletins, calendar invitations, phishing emails, and PR emails. This is just one strand of working life, and yet, it’s entirely possible to be overwhelmed by these emails alone within 5 minutes of starting your day. When you add into the mix newsletter subscriptions, social media, and workplace communication channels, there is just an incredible amount of transitory knowledge. Experts have often pointed towards information overload as the cause of productivity issues, particularly in industries that rely on smooth knowledge sharing. There are roughly 328.77 million terabytes of data created each day, so naturally, there is an awful lot of information to unjumble. Industries across the board are undergoing sweeping digital transformations, with 89% of businesses adopting a digital-first strategy, the sheer amount of information asks a lot of questions on how we can maintain productivity and make informed decisions.Let’s delve into the topic of information overload in the workplace and analyze how, by making adjustments to research content curation, companies can go a long way to limiting its impact The Digital Transformation and Infobesity Workplaces have come a long way in a short space of time. From packed filing cabinets filled with papers to now huge online databases, the digitization of workplaces has undeniably altered the way we access and manage information. While this transformation has its advantages, it has also given rise to the phenomenon of "infobesity,” whereby there is simply too much information being thrown at workers.Forbes identifies infobesity as a formidable challenge, particularly in environments where companies need to make decisions quickly. In fact, the cost of information overload to the US economy is staggering, amounting to nearly $900 billion in annual productivity loss. Some particular industries that are facing an ongoing battle with information overload include e-commerce, where companies such as Procter & Gamble can suffer from analysis paralysis due to the deluge of market research and consumer data, and financial services companies, which handle an extensive array of financial transactions and regulatory data. There are roughly four areas that categorize the information which can be a burden to employees. Repetitive, Irrelevant, Intensive, and Inconsistent information leads to around 4 hours of wasted time in a working week. CEOs and C-suite executives are also feeling the heat of “always being on” and the relentless pressure of managing so much information is a byproduct of that. The solution is getting help with AI-enhanced tools that can efficiently manage and prioritize information flow to alleviate the burden of information overload. AI-Powered Solutions for Content Curation In response to this escalating issue of information overload, innovative solutions leveraging artificial intelligence (AI) are emerging. In the same way as with your favorite streaming platforms such as Netflix and Spotify, AI-based content intelligence systems offers a clear way to address this challenge. These systems are designed to sift through vast amounts of online content, identifying authoritative sources and condensing information into digestible chunks. AI curators can be utilized for workplace learning and knowledge discovery. These AI-powered platforms harness advanced algorithms, which enable users to make informed decisions without being inundated by irrelevant or overwhelming data. The Concept of Anti-search for Streamlining There is a natural propensity from humans to expand their quest for information, but consider the concept of anti-search, which distills content from authoritative sources into bite-sized, structured, and manageable pieces of knowledge. Instead of relying on algorithms to rank content on Google or other search engines, the concept of anti-search is a more considered approach. It takes into account the author’s credibility, the content value, and user context. This gives users genuine insights rather than just generic information.There are plenty of examples of organizations creating workarounds for actually limiting content access to be more streamlined. For example,as stated in an HBR article, the New South Wales Department of Planning and Environment (NSW DPE) recently established an audit for their new intranet. They implemented a quarterly accuracy review and an experience review, which led them to condense 2,000 web pages into just 500.Elsewhere, the “Information Overload Research Group” (IORG) was a non-profit organization dedicated to addressing the issue of information overload. The members included experts from various sectors, including technology companies like Microsoft Research, IBM, and Google. The group’s work focused on understanding the root causes of information overload, such as the rapid growth of connected devices, new communication channels, and globalization, and developing strategies to manage and reduce its impact. Unfortunately, they are no longer in operation, but hopefully, they have laid the groundwork, and similar groups will pop up moving forward. Empowering Lifelong Learning Lifelong learning is something essential for any profession in any walk of life. Lifelong learning through knowledge distillation is not a new concept, educational organizations have been plucking the best resources from the mountain of potential journals and books for centuries. However, these traditional methods naturally struggle to keep pace with the scale and speed of information flow in the digital age. AI-powered solutions bridge this gap by offering knowledge distillation and personalized learning experiences. There is no doubt that Information overload is an inevitable result of our desire to constantly engage with each other in a work setting, but it can lead to employee disengagement and poor decision-making. It’s vital to create a low-burden culture and not pile unnecessary pressure on employees. As organizations strive to navigate this tricky path, AI-driven solutions for content curation and knowledge management emerge as key strategies for combating information overload. Through leveraging advanced algorithms and AI technologies, these solutions offer a path toward a balanced and productive digital workplace. The coexistence of search and anti-search platforms should lead to a future where knowledge is accessible and personalized to individual needs. Using AI intelligently, we can create a future where information overload is no longer a huge barrier to overcome but an opportunity for growth and innovation. Nowadays, we are constantly bombarded with such a high volume of information from various sources. This massive influx of data, coupled with the development of content generation from AI, undoubtedly gives us greater access to knowledge than we’ve ever had before. However, it also brings in the problem of information overload. Nowadays, we are constantly bombarded with such a high volume of information from various sources. This massive influx of data, coupled with the development of content generation from AI, undoubtedly gives us greater access to knowledge than we’ve ever had before. However, it also brings in the problem of information overload. Think of a typical working day for yourself or a colleague. You might open your inbox, which is absolutely stacked with threads, internal bulletins, calendar invitations, phishing emails, and PR emails. This is just one strand of working life, and yet, it’s entirely possible to be overwhelmed by these emails alone within 5 minutes of starting your day. When you add into the mix newsletter subscriptions, social media, and workplace communication channels, there is just an incredible amount of transitory knowledge. Think of a typical working day for yourself or a colleague. You might open your inbox, which is absolutely stacked with threads, internal bulletins, calendar invitations, phishing emails, and PR emails. This is just one strand of working life, and yet, it’s entirely possible to be overwhelmed by these emails alone within 5 minutes of starting your day. When you add into the mix newsletter subscriptions, social media, and workplace communication channels, there is just an incredible amount of transitory knowledge. Experts have often pointed towards information overload as the cause of productivity issues, particularly in industries that rely on smooth knowledge sharing. There are roughly 328.77 million terabytes of data created each day, so naturally, there is an awful lot of information to unjumble. 328.77 million terabytes of data 328.77 million terabytes of data Industries across the board are undergoing sweeping digital transformations, with 89% of businesses adopting a digital-first strategy , the sheer amount of information asks a lot of questions on how we can maintain productivity and make informed decisions. Let’s delve into the topic of information overload in the workplace and analyze how, by making adjustments to research content curation, companies can go a long way to limiting its impact 89% of businesses adopting a digital-first strategy 89% of businesses adopting a digital-first strategy The Digital Transformation and Infobesity Workplaces have come a long way in a short space of time. From packed filing cabinets filled with papers to now huge online databases, the digitization of workplaces has undeniably altered the way we access and manage information. While this transformation has its advantages, it has also given rise to the phenomenon of "infobesity,” whereby there is simply too much information being thrown at workers. Forbes identifies infobesity as a formidable challenge, particularly in environments where companies need to make decisions quickly. In fact, the cost of information overload to the US economy is staggering, amounting to nearly $900 billion in annual productivity loss. Some particular industries that are facing an ongoing battle with information overload include e-commerce, where companies such as Procter & Gamble can suffer from analysis paralysis due to the deluge of market research and consumer data, and financial services companies, which handle an extensive array of financial transactions and regulatory data. There are roughly four areas that categorize the information which can be a burden to employees. Repetitive, Irrelevant, Intensive, and Inconsistent information leads to around 4 hours of wasted time in a working week. CEOs and C-suite executives are also feeling the heat of “always being on” and the relentless pressure of managing so much information is a byproduct of that. The solution is getting help with AI-enhanced tools that can efficiently manage and prioritize information flow to alleviate the burden of information overload. AI-Powered Solutions for Content Curation In response to this escalating issue of information overload, innovative solutions leveraging artificial intelligence (AI) are emerging. In the same way as with your favorite streaming platforms such as Netflix and Spotify, AI-based content intelligence systems offers a clear way to address this challenge. These systems are designed to sift through vast amounts of online content, identifying authoritative sources and condensing information into digestible chunks. AI curators can be utilized for workplace learning and knowledge discovery. These AI-powered platforms harness advanced algorithms, which enable users to make informed decisions without being inundated by irrelevant or overwhelming data. The Concept of Anti-search for Streamlining There is a natural propensity from humans to expand their quest for information, but consider the concept of anti-search, which distills content from authoritative sources into bite-sized, structured, and manageable pieces of knowledge. Instead of relying on algorithms to rank content on Google or other search engines, the concept of anti-search is a more considered approach. It takes into account the author’s credibility, the content value, and user context. This gives users genuine insights rather than just generic information. There are plenty of examples of organizations creating workarounds for actually limiting content access to be more streamlined. For example, as stated in an HBR article , the New South Wales Department of Planning and Environment (NSW DPE) recently established an audit for their new intranet. They implemented a quarterly accuracy review and an experience review, which led them to condense 2,000 web pages into just 500. Elsewhere, the “Information Overload Research Group” (IORG) was a non-profit organization dedicated to addressing the issue of information overload. The members included experts from various sectors, including technology companies like Microsoft Research, IBM, and Google. as stated in an HBR article as stated in an HBR article The group’s work focused on understanding the root causes of information overload, such as the rapid growth of connected devices, new communication channels, and globalization, and developing strategies to manage and reduce its impact. Unfortunately, they are no longer in operation, but hopefully, they have laid the groundwork, and similar groups will pop up moving forward. Empowering Lifelong Learning Lifelong learning is something essential for any profession in any walk of life. Lifelong learning through knowledge distillation is not a new concept, educational organizations have been plucking the best resources from the mountain of potential journals and books for centuries. However, these traditional methods naturally struggle to keep pace with the scale and speed of information flow in the digital age. AI-powered solutions bridge this gap by offering knowledge distillation and personalized learning experiences. There is no doubt that Information overload is an inevitable result of our desire to constantly engage with each other in a work setting, but it can lead to employee disengagement and poor decision-making. It’s vital to create a low-burden culture and not pile unnecessary pressure on employees. As organizations strive to navigate this tricky path, AI-driven solutions for content curation and knowledge management emerge as key strategies for combating information overload. Through leveraging advanced algorithms and AI technologies, these solutions offer a path toward a balanced and productive digital workplace. The coexistence of search and anti-search platforms should lead to a future where knowledge is accessible and personalized to individual needs. Using AI intelligently, we can create a future where information overload is no longer a huge barrier to overcome but an opportunity for growth and innovation.

Information Overload: How AI With Research Content Curation Could Lead the Way

Stories, News & Mentions about CURA

Cura’s Stories on HackerNoon