Tsela ya go Master Ditshekatsheko tša Nako ya Kgonthe Ka AWS: Nako ya Moela le ka Ntle ga yona

In my last article , re ile ra ahla-ahla kgolo e kgolo ya ditiragalo lefaseng la lehono leo le laolwago ke data. Ka didirišwa tše dintši gakaakaa, didirišwa tše bohlale le metšhene gohle go re dikologa, tekanyo ya data yeo e hlotšwego ke e kgolo kudu. Re ile ra hlahloba gape ka fao sefala sa mmino wa diletšo se ka thušago go iša ditiragalo tše go dikgopelo tše di nepagetšego. Le ge go le bjalo, go tliša ditiragalo go ka no se lekane gore dikgwebo di dire khuetšo.

Ka go sekaseka ditiragalo tše go kwešiša boitshwaro bja badiriši, dikgwebo di ka hlankela bareki ba tšona gakaone ka go tšea diphetho tše bohlale. Sefala sa ditshekatsheko tša nako ya nnete se ka thuša go fetolela ya data ya tiragalo go bohlale bjo bo nago le mohola.

Sehlogo se se hlahloba ka moo o ka agago sefala sa ditshekatsheko tša nako ya nnete ka go šomiša AWS, go sekaseka ditharollo tše di kgonegago, le go fa tlhahlo ya kgato ka kgato ya go phethagatša sefala seo se ka lekanyetšwago le seo se ka botwago. Go aga sefala se go akaretša megato ye meraro: go tsenya datha, go šoma le go botšiša. Ditshekatsheko tša nako ya nnete gantši di lebeletše kudu ditshekamelo le dipaterone ge nako e dutše e eya - e ka ba boitshwaro bja mosediriši goba tshepedišo ya tshepedišo.

Datha ya lelokelelo la nako ka tlhago e rulaganya ditiragalo ka tatelano, e dira gore go be bonolo go sekaseka ya data go tloga motsotsong o mongwe go ya go o mongwe. Polokelo ya letoto la nako e sepelelana gabotse le tlhokego ye, e dumelela dikgopelo go bala dimetriki. AWS e fana ka didirišwa tša go swana le SQS, Lambda, Timestream, le Quicksight tšeo di šomago mmogo ka ntle le mathata go aga sefala se.

Go na le dikarolo tše tharo tše kgolo tšeo di akaretšwago go ageng sefala sa ditshekatsheko tša nako ya nnete

• Data Ingestion : Ditiragalo tšeo di phatlaladitšwego go tšwa go dikgopelo di ka elela ka gare ga AWS ka Amazon Kinesis goba ka go lebanya go SQS.

• Tshepetšo ya Datha : AWS Lambda e ka šoma ya data gomme ya e ngwala go Timesream.

• Visualization & Querying : Custom dashboards kapa Quicksight ka kopanya le Timestream bakeng sa pono le temogo

  
  

  Seswantšho se se lego ka mo tlase se bontšha sebopego sa sefala sa ditshekatsheko

  
  

Moela wa nako

Amazon Timestream , polokelo ya tshedimošo ya lelokelelo la nako la AWS, e reretšwe go fihlelela ditlhohlo tša go šoma le go sekaseka palo e kgolo ya datha ka bokgoni. Timestream ke serverless, scalable, ebile e loketše dikgopelo tšeo di nyakago ditshekatsheko tša datha tša nako ya nnete. Dikarolo tša yona tše bohlokwa di akaretša:

• Go lekanya ka go itiriša : E ka šoma ditiragalo tše dimilione tše dikete ka letšatši, go lekanya ka go itiriša go fihlelela nyakego.
• Performance : E fana ka ho fihlela ho makhetlo a 1000 ka potlako potso tshebetso bapisoa le polokelongtshedimosetso kamano.
• Serverless : E laolwa ka botlalo ke AWS, go fokotša tšhelete ya godimo.
• Smart Storage Tiers : Optimized polokelo ditiers bakeng sa morao tjena (ka memoring ya) le histori (poloko makenete) ya data.
• Thekgo ya dipotšišo tša SQL : E thekga dipotšišo tša SQS bakeng sa go dira dipotšišo tše di raraganego, dikopano, le ditshekatsheko tša letoto la nako.
• Dikopanyo : E thekga kopanyo ye e se nago mathata le ditirelo tše dingwe tša AWS.

Phethagatšo

Thempleite ya go bopa leru (CFN) ya methopo ka moka ye e nyakegago e ka hwetšwa go Github Repo .

• Go metša ga Datha

  Ditirelo di ka phatlalatša ditiragalo tša maleba go mothaladi wa SQS, wo o šomago bjalo ka mothaladi wa molaetša wa tshepedišo. Go godiša go botega, mothaladi wa ditlhaka tše di hwilego (DLQ) o beakantšwe go bapa le mothaladi wa mathomo wa SQS. Ditiragalo tšeo di palelwago ke go šongwa di ya go DLQ ka morago ga moedi wa go leka gape. Ditiragalo tše go ya pele di ka šomišwa go lokiša diphošo, go swara go palelwa, le dinyakišišo.

  
  

  Ka fase ke seripa sa thempleite ya CFN go hlama mothaladi wa SQS le DLQ ya yona ye e amanago le yona:

  
  

   EventQueueDLQ: Description: 'An DLQ queue that receives failed events' Type: AWS::SQS::Queue Properties: FifoQueue: true ContentBasedDeduplication: true QueueName: !Ref DLQQueueName SqsManagedSseEnabled: true VisibilityTimeout: 240 EventQueue: Description: 'An SQS queue that receives events' Type: 'AWS::SQS::Queue' Properties: QueueName: !Ref SQSQueueName FifoQueue: true ContentBasedDeduplication: true KmsMasterKeyId: alias/aws/sqs VisibilityTimeout: 240 RedrivePolicy: deadLetterTargetArn: !Sub ${EventQueueDLQ.Arn} maxReceiveCount: 5

• Tshepedišo ya datha

  Mošomo wa AWS Lambda o beakantšwe bjalo ka modiriši wa datha, yo a ikarabelago ka go swara le go šoma ditiragalo tšeo di phatlaladitšwego mothalong wa SQS. E kgoromeletša ditiragalo bjalo ka dimetriki go polokelo ya tshedimošo ya Timestream ka morago ga tshepedišo.

  
  

  Ka tlase ke seripa sa thempleite ya CFN ya Lambda le go beakanya ga yona go SQS:

  
  

   EventProcessorLambda: Type: 'AWS::Lambda::Function' Description: 'Lambda function that processes events from the SQS queue and writes to Timestream.' Properties: FunctionName: !Ref LambdaFunctionName Handler: 'index.lambda_handler' Role: !GetAtt LambdaExecutionRole.Arn Runtime: 'python3.12' MemorySize: 1024 Timeout: 120 Environment: Variables: TIMESTREAM_DATABASE_NAME: !Ref EventsDatabaseName TIMESTREAM_TABLE_NAME: !Ref EventsTableName Code: ZipFile: | # Lambda function code goes here SQSToLambdaEventSourceMapping: Type: 'AWS::Lambda::EventSourceMapping' Description: 'Maps the SQS queue as the event source for the Lambda function.' Properties: BatchSize: 10 EventSourceArn: !GetAtt EventQueue.Arn FunctionName: !GetAtt EventProcessorLambda.Arn Enabled: 'True'

  
  

• Lebenkele la Datha

  Amazon Timestream e šoma bjalo ka lebenkele la mathomo la datha bakeng sa ditiragalo ka moka tšeo di tšweleditšwego go ralala le ditirelo tše di fapa-fapanego. Thulaganyo e akaretša:

  • Database : E šoma bjalo ka setshelo se se kwagalago sa tafola e tee goba tše ntši
  • Tafole : Ka gare ga polokelo ya tshedimošo, ditafola di boloka datha ya nnete ya dimetriki

Ka fase ke seripa sa thempleite ya CFN go hlama polokelo ya tshedimošo ya Timestream le tafola:

 EventsDatabase: Description: 'Timestream database to store event data' Type: 'AWS::Timestream::Database' Properties: DatabaseName: !Ref EventsDatabaseName KmsKeyId: alias/aws/timestream EventsTable: Description: 'Timestream table that stores event metrics' Type: 'AWS::Timestream::Table' DependsOn: EventsDatabase Properties: DatabaseName: !Ref EventsDatabase TableName: !Ref EventsTableName RetentionProperties: MemoryStoreRetentionPeriodInHours: 72 MagneticStoreRetentionPeriodInDays: 365

• Pono & Go Botšiša

  • Potšišo

    Timestream e fana ka khonsule ya Potšišo yeo e dumelelago badiriši go sepetša dipotšišo kgahlanong le tafola, mohlala:

     -- Get the 10 most recent metrics in the past 15 minutes. SELECT * FROM "events-db"."events-metrics" WHERE time between ago(15m) and now() ORDER BY time DESC LIMIT 10

  
  

  • Go bona dilo ka leihlo la kgopolo

    AWS e fana ka dikopanyo tše ntši tša ka ntle ga lepokisi (mohlala, QuickSight , Grafana ) le Timestream, e dira gore go be bonolo go sekaseka, go bona ka leihlo la kgopolo, le go hwetša ditemogo.

Go dira diteko

Ditirelo di ka phatlalatša tiragalo ka sebopego se se latelago go SQS yeo e hlohleletšago phallo ka moka ya tshepedišo:

 { "order_id": "test-order-1", "customer_id": "test-customer-1", "event_type": "order_success", "metric_value": 1 }

Mafetšo

Sebopego se se fa tsela ye bonolo le ye e šomago gabotse ya go aga sefala sa ditshekatsheko seo se ka lekanyetšwago le seo se ka botwago. Go na le mekgwa ye mengwe go ya ka dinyakwa tše itšego, go akaretšwa AWS Kinesis Streams bakeng sa tshepedišo ya ditiragalo, Prometheus bakeng sa lebenkele la data, le S3+Athena bakeng sa tshepedišo ya sehlopha le ditshekatsheko.