Imayna chiqa pacha t'aqwiyta AWS kaqwan kamachiy: Pacha puriy chaymanta aswan karu

Qhipa qillqasqaypi , kunan pacha willaykunawan purichisqa pachakunapi ruwaykuna exponencial wiñayninmanta rimarqayku. Askha ruwanakuna, yuyaysapa dispositivokuna, makinakuna tukuy muyuriqniykupi kaptin, mayk'a willay ruwasqakuna hatun kanku. Hinallataq, imayna huk orquestación plataforma yanapanman kay ruwaykuna allin ruwanakunaman chayachiyta maskarqaniku. Ichaqa, eventokuna apachiyqa manapaschá suficientechu kanman negociokuna huk impacto ruwanankupaq.

Kay ruwaykunata t'aqwispa, llamk'achiqkunap ruwayninta hamut'anapaq, negociokuna aswan yachaysapa tanteayta ruwaspa rantiqninkunaman aswan allinta yanapayta atinku. Chiqa pacha t'aqwiy plataforma yanapakuy atikunman tikrayta ruwaypa willayta sumaq yuyayman.

Kay qillqa imayna huk chiqa pacha t'aqwiy plataforma ruwayta maskan AWS kaqwan, atikuq allichaykunata chaninchaspa, chaymanta huk llamkanamanta llamkanaman pusayta qun huk escalable chaymanta atikuq plataforma ruwanapaq. Kay plataforma ruwayqa kimsa ruwaykunata ruwan: willayta mikhuy, ruway, tapuy ima. Chiqa pacha t'aqwiyqa sapa kuti tendenciakuna chaymanta ruwanakuna pachawan t'inkikun - sichus ruwaqpa ruwaynin utaq sistema ruwaynin.

Tiempo-serie nisqa willakuykunaqa naturalmentem sucesokunata qatiqninpi organizan, chaymi mana sasachu datoskunata huk ratomanta huk ratoman t'aqwiy. Tiempo-serie waqaychayqa kay necesidadwan allinta tupan, ruwanakuna mitricakuna yupayta saqin. AWS SQS, Lambda, Timestream chaymanta Quicksight hina yanapakuykunata qun chaymanta mana pantasqa kuska llamk'anku kay plataforma ruwanapaq.

Kimsa hatun rakikuna kanku chiqa pacha t'aqwiy plataforma ruwaypi

• Willayta Mikhuy : Ruwaykunamanta lluqsisqa ruwaykuna AWS kaqman Amazon Kinesis kaqninta utaq chiqalla SQS kaqman puriyta atinku.

• Willayta Ruway : AWS Lambda willayta ruwayta atin chaymanta Timesream kaqman qillqayta atin.

• Rikuchiy & Tapuy : Sapanchasqa tablakuna utaq Quicksight Timestream kaqwan tinkiyta atinku qhawaypaq chaymanta qhawaykunapaq

  
  

  Uraypi siq'iqa t'aqwiy plataformap arquitecturanta rikuchin

  
  

Pacha puririynin

Amazon Timestream , AWS kaqpa pacha-serie willaypa tiyapuynin, ruwasqa kachkan sasachakuykunata ruwanapaq chaymanta hatun willaykunata allinta t'aqwiypaq. Timestream mana servidorniyuq, escalable chaymanta allin ruwanakuna chiqa pacha willay t'aqwiyta mañaq. Paypa hatun ruwayninkunaqa kayhinam:

• Kikinmanta hatunyachiy : Sapa p'unchawpi hunu waranqa waranqa ruwaykunata ruwayta atin, kikinmanta hatunyachiy mañakuy hunt'anapaq.
• Ruway : 1000 kutikama aswan utqaylla tapuy ruwayta qun relacional willay tantanakunawan tupachisqa.
• Mana servidorniyuq : AWS kaqwan tukuyninpi kamachisqa, hawa qullqita pisiyachispa.
• Smart Storage Tiers : Allinchasqa waqaychay patakuna qhipa (yuyariypi) chaymanta ñawpaqmanta (magnético waqaychay) willaykunapaq.
• SQL tapuy yanapakuy : SQS tapuykunata yanapan sasa tapuykunata, huñusqakunata, chaymanta pacha-serie t'aqwiykunata ruwanapaq.
• Huñunakuykuna : Yanapakun mana pantasqa tinkiyta wak AWS yanapakuykunawan.

Implementación nisqa

Tukuy necesitasqa yanapakuykunapaq puyu ruway (CFN) plantilla tarikunman Github Repo kaqpi.

• Datos Ingestion nisqa

  Yanapakuykuna tupaq ruwaykunata SQS filaman lluqsichiyta atinku, mayqinchus llamkanapaq willakuy fila hina llamkan. Confiabilidad aswan allin kananpaq, huk wañusqa qillqa fila (DLQ) ñawpaq SQS filawan kuska ruwasqa. Mana ruwasqa ruwaykuna DLQ kaqman rinku wakmanta kallpachakuy umbral kaqmanta. Kay ruwaykuna aswanta llamk'achiy atikunman pantasqa allichaypaq, pantasqakuna allichaypaq chaymanta investigacionkunapaq.

  
  

  Uraypi huk fragmento CFN plantillamanta kachkan SQS filata chaymanta chaywan tinkisqa DLQ ruwanapaq:

  
  

   EventQueueDLQ: Description: 'An DLQ queue that receives failed events' Type: AWS::SQS::Queue Properties: FifoQueue: true ContentBasedDeduplication: true QueueName: !Ref DLQQueueName SqsManagedSseEnabled: true VisibilityTimeout: 240 EventQueue: Description: 'An SQS queue that receives events' Type: 'AWS::SQS::Queue' Properties: QueueName: !Ref SQSQueueName FifoQueue: true ContentBasedDeduplication: true KmsMasterKeyId: alias/aws/sqs VisibilityTimeout: 240 RedrivePolicy: deadLetterTargetArn: !Sub ${EventQueueDLQ.Arn} maxReceiveCount: 5

• Willayta ruway

  AWS Lambda ruwana willay ruwaq hina ruwasqa kachkan, SQS filaman lluqsisqa ruwaykunata hapinapaq chaymanta ruwanapaq. Ruwaykunata mitricakuna hina Timestream willay tantanaman tanqan ruwaymanta qhipaman.

  
  

  Uraypi huk fragmento plantilla CFN Lambdapaq chaymanta mapeo SQSman:

  
  

   EventProcessorLambda: Type: 'AWS::Lambda::Function' Description: 'Lambda function that processes events from the SQS queue and writes to Timestream.' Properties: FunctionName: !Ref LambdaFunctionName Handler: 'index.lambda_handler' Role: !GetAtt LambdaExecutionRole.Arn Runtime: 'python3.12' MemorySize: 1024 Timeout: 120 Environment: Variables: TIMESTREAM_DATABASE_NAME: !Ref EventsDatabaseName TIMESTREAM_TABLE_NAME: !Ref EventsTableName Code: ZipFile: | # Lambda function code goes here SQSToLambdaEventSourceMapping: Type: 'AWS::Lambda::EventSourceMapping' Description: 'Maps the SQS queue as the event source for the Lambda function.' Properties: BatchSize: 10 EventSourceArn: !GetAtt EventQueue.Arn FunctionName: !GetAtt EventProcessorLambda.Arn Enabled: 'True'

  
  

• Willayta Waqaychay

  Amazon Timestream ñawpaq willay waqaychasqa hina llamkan llapa ruwaykuna imaymana yanapakuykunapi ruwasqa. Chay churayqa kaykunatam ruwan:

  • Willayta waqaychana : Huk icha aswan tablakunapaq lógico waqaychana hinam llamkan
  • Tabla : Willayta waqaychana ukhupi, tablakuna chiqap mitrica willayta waqaychan

Uraypi CFN plantillapaq t'aqa Timestream willay tantanata chaymanta tablata ruwanapaq:

 EventsDatabase: Description: 'Timestream database to store event data' Type: 'AWS::Timestream::Database' Properties: DatabaseName: !Ref EventsDatabaseName KmsKeyId: alias/aws/timestream EventsTable: Description: 'Timestream table that stores event metrics' Type: 'AWS::Timestream::Table' DependsOn: EventsDatabase Properties: DatabaseName: !Ref EventsDatabase TableName: !Ref EventsTableName RetentionProperties: MemoryStoreRetentionPeriodInHours: 72 MagneticStoreRetentionPeriodInDays: 365

• Rikuchiy & Tapukuy

  • Tapuy

    Timestream huk Tapuy consola qun chaymanta ruwaqkunata tapuykunata tablapa contranpi purichiyta saqin, kayhina:

     -- Get the 10 most recent metrics in the past 15 minutes. SELECT * FROM "events-db"."events-metrics" WHERE time between ago(15m) and now() ORDER BY time DESC LIMIT 10

  
  

  • Rikuchiy

    AWS achka mana qutu tinkiykuna qun (kayhina QuickSight , Grafana ) Timestream kaqwan, chaymanta mana sasachu t'aqwiy, qhaway chaymanta hamut'aykunata hurquy.

Prueba ruway

Serviciokuna ruwayta kay formatopi SQS kaqman lluqsichiyta atinku mayqinchus tukuy ruwana puriyta llamk'achin:

 { "order_id": "test-order-1", "customer_id": "test-customer-1", "event_type": "order_success", "metric_value": 1 }

Conclusion

Kay arquitectura huk sanu chaymanta allin ñanta qun huk escalable chaymanta confiable analisis plataforma ruwanapaq. Huk alternativakuna kanku sapanchasqa necesidadkunamanta, chaymanta AWS Kinesis Streams ruwaypaq, Prometheus huk willay waqaychasqapaq, chaymanta S3+Athena huñu ruwanapaq chaymanta t'aqwiypaq.