Python - Django: Esengeli Jamais Otia Biloko Oyo Na Transactions

Makambo eza na kati

• Part 1 Bokuse - Plongée profonde na ba Transactions ya Django
• TLDR, Bokuse ya koloba
• Ndenge nini ba transactions ekoki kobebisa ba apps na biso?
• Na
• Ezali na likama
• Libanda
• Na sima - Ndenge nini ba commandes ya PSQL ezo bloquer moko na mosusu
• Maziba oyo euti na yango

Part 1 Résumé - Plongée Profonde Na Ba Transactions Ya Django

Na post ya liboso , toyekolaki nini esalemaka ntango fonction oyo ekembisami na transaction.atomic ebengami mpe nini esalemaka na with transaction.atomic() . Na bokuse:

1. Connexion na DB esalemi to ezuami.
2. Transaction esalemi, ndakisa, BEGIN; etindamaka na DB (soki DB ezali PSQL to variante mosusu ya SQL).
3. Kobanda sikoyo tii tango fonction ekozala to déclaration with ekosila - ezala na erreur to na succès - tokozala na transaction, pe DB ekozala kozela biso.
4. Yango elingi koloba ete (ata moke te mpo na ba DB ACID lokola PSQL) transaction ekosimba ba blocages na ba tableaux mpe ba lignes oyo ezali ko changer.
5. Tango opération ya DB ya Django esalemi, DB esimbaka blocage oyo ekokani mpe epekisaka ba opérations nionso oyo ezali na conflit na tableau to molongo wana.
6. Yango ekoki kosala ete ba connexions mosusu esila mpo na kozela ete blocage ebima.
7. Soki opération elongi te to soki erreur ya temps d'exécution ezali, DB ezo zongisa sima transaction mobimba pe ezo bimisa ba verrouillages.
8. Soki transaction mobimba elongi, alors ba changements nionso e committé pe disponible na ba connexions misusu ya DB.
9. Ba serrures ebimisami.

Sikoyo, toko lobela nini tosengeli kotia na transaction mpe nini tokoki koboya. Na tina ya bizaleli ya transaction oyo elandi, ba opérations mosusu ezali dangereuse tango etie na bloc ya transaction.

• Ba transactions esimbaka ba serrures tii tango transaction ekozanga to ekosila.
• Ba transactions ezongisaka sima ba opérations na yango nionso ya DB tango elongi te.
• Ba blocage ya demande ya ba transactions noki soki opération ya DB esalemi.

TLDR, Bokuse ya koloba

Na

• Ba opérations reversibles na DB - Soki opération ya DB ekoki ko reverser te mpe transaction elongi te, DB ekotikala na état ya mabe mpo transaction ekomeka ko roll back ba changements automatiquement, kasi ezali irreversible, yango wana eko échouer.
• Ba opérations liées na DB oyo esengeli - Tokoki kosala dossier ya compte bancaire ya sika te sans dossier ya usager, yango wana esengeli tosala nionso mibale na transaction moko.
• Logique d’affaires réversible mpe oyo etali yango - Ko calculer solde total oyo client azali na yango sima ya kosala enregistrement ya sika ya compte na dépôt (même oyo ekoki kobima na transaction na modélisation mpe coordination ya mayele).

Ezali na likama

• Ba requêtes ya malembe - Kozua ba données ezalaka mingi mingi mbangu, longola se ba scénarios misato oyo. Ba requêtes oyo eko ralentir transaction mpe eko prolonger temps oyo esimbaka ba serrures, oyo ekozala na ba effets ya mabe na ba usagers misusu.
• Misala na ba tableaux ebele - Transaction na ba opérations na ba tableaux ebele ekoki ko bloquer tableau moko na moko tii ekosalema. Yango ezali mingi mingi na ba migrations ya Django - raison mosusu ya ko garder ba migrations mike pe ko focuser na tableau moko to mua ba tableaux na mbala moko.
• Migrations ya ba données - Lokola ba transactions esimbaka na blocage kobanda tango query esalemi tii tango transaction ekozanga to ekosila, migration oyo esalaka na ligne nionso na tableau ekosuka na ko bloquer tableau mobimba, soit na kopekisa ba lectures to ba écritures na ligne moko na moko.
• (Mpo na PSQL mpe SQLite kaka*) Kobongola ba tableaux to ba colonne - ba opérations oyo esengaka lolenge ya makasi ya ba blocages mpe, na yango, ekopekisa kotanga/kokoma na tableau mobimba. Ba opérations wana nde ekoki kosala que panne ezala mingi.

Libanda

• Ba opérations irreversibles - Nionso oyo ezali na transaction esengeli ezala reversible soki transaction ezo zonga sima ; soki totie appel ya API na transaction, ekoki kozala undone te.
• Blocage ya ba appels - Lokola ba transactions epekisaka ba requêtes nionso misusu esala na ba tableaux/lignes oyo transaction ezali ko changer, code nionso oyo ematisaka durée ya transaction ekosala que DB ezala bloqué, ekosala que ba temps d’extraits mpe inresponsabilité na ba apps oyo etali DB.

Koba kotanga mpo na ba alternatives mpe bandakisa ya code.

Ndenge nini ba transactions ekoki kobebisa ba apps na biso?

Risque ya liboso ezali que ba transactions esimba ba serrures tii ekosalema pona kopekisa ba opérations conflits na ba tableaux na ba lignes pe ko permettre transaction ezala reversible - yango ezali essentiel pona kosala que ba opérations ya DB na transaction ezala atomique. Yango elingi koloba transaction ya kala oyo esalaka na ba tableaux ebele to mwa ba critiques ekoki kosala ba pannes na ko hogging ba serrures mpe kopekisa kotanga/kokoma na ba tableaux/milɔngɔ wana.

En essence, soki totie code ya mabe na bloc ya transaction, tokoki efficacement ko kitisa DB na ko bloquer ba connexions nionso misusu na DB esala ba opérations likolo na yango.

Risque secondaire ezali que ba transactions esengeli ezala reversible mpe ESENGELI kozala reversible. DB ezongisaka sima automatiquement opération nionso soki erreur esalemi na transaction. Yango wana, ba opérations ya DB oyo totie na transaction esengeli ezala reversible - pona mingi, tozali na besoin te ya komitungisa pona likambo oyo na PSQL. Kasi ezali boni mpo na ba code mosusu?

Mbala mingi, tango tobongoli ba données na biso, esengeli tosala misala ya bolandi lokola ko tirer ba événements, ko mettre à jour ba services, kotinda ba notifications ya push, etc. Misala oyo ezali reversible TE - tokoki te ko désend événement, demande, to notification. Soki libunga esalemi, mbongwana ya ba données ezongaka sima, kasi tosi totindi notification ya push oyo elobi, "Rapport na yo esalemi; finá awa mpo na kotala yango." Nini esalemaka ntango mosaleli to ba services mosusu esalaka na ba informations wana ya lokuta? Ekozala na cascade ya ba échecs. Yango wana, code nyonso oyo ekoki kozongisama nsima te esengeli te kozala na mombongo, to tozali na likama ya kotika système na biso na ezalela ya mabe ntango libunga esalemi na mombongo.

Na

• Ba opérations reversibles na DB - Soki opération ya DB ekoki ko reverser te mpe transaction elongi te, DB ekotikala na état ya mabe mpo transaction ekomeka ko roll back ba changements automatiquement, kasi ezali irreversible, yango wana eko échouer.
• Ba opérations liées na DB oyo esengeli - Tokoki kosala dossier ya compte bancaire ya sika te sans dossier ya usager, yango wana esengeli tosala nionso mibale na transaction moko.
• Logique d’affaires réversible mpe oyo etali yango - Ko calculer solde total oyo client azali na yango sima ya kosala enregistrement ya sika ya compte na dépôt (même oyo ekoki kobima na transaction na modélisation mpe coordination ya mayele).

Ezali na likama

Wana ezali makambo oyo, selon combien de données ezali ko traité mpe trafic ya DB, ekoki kosala ba pannes mpo na kokanga ba serrures pendant trop. Makambo oyo nyonso ezali malamu soki ezwi ntango molai te.

• Ba queries ya malembe — Kozua ba données ezalaka mingi mingi mbangu, longola kaka ba scénarios misato oyo. Ba requêtes oyo eko ralentir transaction mpe eko prolonger temps oyo esimbaka ba serrures, oyo ekozala na ba effets ya mabe na ba usagers misusu.

  • Bandakisa
    • Mituna na makonzí oyo ezali na index te
    • Mituna na ba tableaux ya minene
    • Azali kosangana

   @transaction.atomic def process_large_order_report(start_date, end_date, min_order_value=1000): # Complex query with multiple joins and aggregations large_orders = Order.objects.filter( created_at__range=(start_date, end_date), total_amount__gte=min_order_value, status='completed' ).select_related( 'customer', 'shipping_address', 'billing_address' ).prefetch_related( 'items__product__category', 'items__product__supplier' ).annotate( item_count=Count('items'), total_weight=Sum('items__product__weight'), discount_percentage=F('discount_amount') * 100 / F('total_amount') ).filter( # Additional complex filtering Q(customer__user__is_active=True) & (Q(items__product__category__name='Electronics') | Q(items__product__category__name='Furniture')) & ~Q(shipping_address__country='US') ).order_by('-total_amount') # do the transactional work with the large_orders queryset

  • Mosusu
    • Sala ba queries oyo avant pe libanda ya transaction.

 # fixed def process_large_order_report(start_date, end_date, min_order_value=1000): # Complex query with multiple joins and aggregations large_orders = Order.objects.filter( created_at__range=(start_date, end_date), total_amount__gte=min_order_value, status='completed' ).select_related( 'customer', 'shipping_address', 'billing_address' ).prefetch_related( 'items__product__category', 'items__product__supplier' ).annotate( item_count=Count('items'), total_weight=Sum('items__product__weight'), discount_percentage=F('discount_amount') * 100 / F('total_amount') ).filter( # Additional complex filtering Q(customer__user__is_active=True) & (Q(items__product__category__name='Electronics') | Q(items__product__category__name='Furniture')) & ~Q(shipping_address__country='US') ).order_by('-total_amount') # Start the transaction block with transaction.atomic(): # do the transactional work with the large_orders queryset

• Misala na ba tableaux ebele — transaction na ba opérations na ba tableaux ebele ekoki ko bloquer tableau moko na moko tii ekosalema. Yango ezali mingi mingi na ba migrations ya Django — raison mosusu ya kobatela ba migrations moke mpe ko focuser na tableau moko to mua ba tableaux na mbala moko.

  
  

  • Bandakisa
    • Talá Kobongola ebongiseli ya tableau to ya makonzí

   class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] # too many operations operations = [ migrations.RemoveField("Author", "age"), migrations.AddField("Author", "rating", models.IntegerField(default=0)), migrations.AlterField("Book", "price", models.DecimalField(max_digits=5, decimal_places=2)), ]

  • Mosusu
    • Kabola transaction na ba transactions ya mike ebele pe chaîner yango na ba callbacks ya oncommit.

 # fixed # 1st migration class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] operations = [ migrations.RemoveField("Author", "age"), ] # 2nd migration class Migration(migrations.Migration): dependencies = [("migrations", "0002_initial")] operations = [ migrations.AddField("Author", "rating", models.IntegerField(default=0)), ] # 3rd migration class Migration(migrations.Migration): dependencies = [("migrations", "0003_initial")] operations = [ migrations.AlterField("Book", "price", models.DecimalField(max_digits=5, decimal_places=2)), ]

• Migrations ya ba données — Lokola ba transactions esimbaka na blocage kobanda tango query esalemi tii transaction ekozanga to ekosila, migration oyo esalaka na ligne nionso na tableau ekosuka na ko bloquer tableau mobimba, soit na kopekisa kotanga to kokoma na molongo moko na moko.

   def migrate_user_profiles(): # Get all users with legacy profiles users_with_profiles = User.objects.filter( legacy_profile__isnull=False ).select_related('legacy_profile') # Process all users in a single transaction with transaction.atomic(): # Track progress total = users_with_profiles.count() print(f"Migrating {total} user profiles...") # Process each user for i, user in enumerate(users_with_profiles): if i % 100 == 0: print(f"Processed {i}/{total} profiles") legacy = user.legacy_profile legacy.update_new_user_profile()

  • Mosusu
    • Envelopper transaction zinga zinga ya ba opérations individuelles, kasi migration mobimba te. Na kotiaka ba mises à jour na molongo moko na moko na transaction, tosimbaka kaka ba serrures mpo na mwa ntango mokuse.

 # fixed def migrate_user_profiles(): # Get all users with legacy profiles users_with_profiles = User.objects.filter( legacy_profile__isnull=False ).select_related('legacy_profile') # Process all users in a single transaction # Track progress total = users_with_profiles.count() print(f"Migrating {total} user profiles...") # Process each user for i, user in enumerate(users_with_profiles): if i % 100 == 0: print(f"Processed {i}/{total} profiles") with transaction.atomic(): legacy = user.legacy_profile legacy.update_new_user_profile()

• (Mpo na PSQL mpe SQLite kaka *) Kobongola ba tableaux to ba colonne — ba opérations oyo esengaka lolenge ya makasi ya ba blocages mpe, na yango, ekopekisa kotanga/kokoma na tableau mobimba. Ba opérations wana nde ekoki kosala que panne ezala mingi.
  • Ndakisa
    • Bobongola Kolona
    • Bobongola Tableau
  • Mosusu

    • Kima bango na nsima. Ba requêtes oyo ezali nécessaire, MAIS il faut te to diriger yango na heures d'affaires. Politiki ya malamu awa ezali ya kokitisa likama ya bokati na koyeba ndenge nini tableau ezali na ntina mingi, kokanisa ntango boni bopanzani ekoki kozwa, kosala bopanzani tango DB ezali na trafic moke, mpe kobongisa mwango ya bozongisi sima.

    • Kitisa ntango oyo mombongo yango esɛngaka. Yango ekoki kosalema na kokabola tableau mpe kosala migration na ba partitions moko moko. Partitions ya PSQL & Django

      
      

*Django ezo envelopper kaka ba transactions autour ya ba migrations pona PSQL na SQLite.

 class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] # this migration, if on a large table, can slow down and block other operations # do it later operations = [ migrations.RemoveField("Users", "middle_name"), ]

Libanda

• Misala oyo ekoki kozongisama te — Nyonso oyo ezali na kati ya mombongo esengeli kozala oyo ekoki kozongisama nsima soki mombongo yango ezongisami nsima; soki totie appel ya API na transaction, ekoki kozala undone te.

  • Bandakisa
    • Kobakisa likambo na molongo — ko déclencher makambo mosusu
    • Kotinda API Call — mises à jour ya état, misala ya déclencheur, etc.

   def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) async_notification_service.send_email(user.email, "You can login now!") Account.objects.create(user=user, balance=0) # rest of user creation proccess

  • Ba alternatives ya kosala
    • Sala logique ya mombongo ya ntina oyo esengeli kosalema na callback ya oncommit ya transaction — callback ya oncommit ebengamaka TOUJOURS sima ya succès ya transaction, mpe ba mises à jour nionso oyo ewutaka na transaction ezali na callback ya oncommit.
    • Tokoki pe kosala que opération ezala reversible, c'est-à-dire, kosala moyen ya ko effacer ba événements to kotinda appel ya API ya kozongisa sima — Oyo ezali mosala nontrivial pona équipe oyo ezali na molongo ya événement to API. Nazali ko recommander yango te.

 def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess # the transaction is still in progress, so it can still be rolled back, it is not # committed until the transaction block is exited, so putting the notification here # is not a good idea - especially if the job starts immediately tries to read the data # this creates a race condition async_notification_service.send_email(user.email, "You can login now!") def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess transaction.on_commit(partial(async_notification_service.send_email, user.email, "You can login now!"))

• Ko bloquer ba appels — lokola ba transactions epekisaka ba requêtes nionso misusu esala na ba tableaux/milɔngɔ oyo transaction ezali ko changer, code nionso oyo ematisaka durée ya transaction ekosala que DB ezala bloqué, ekosala que ba temps d’extraits mpe inresponsabilité na ba apps oyo etali DB.

  
  

  • Bandakisa
    • Ba appels ya réseau — Kotinda ba demandes na ba API pona kozua ba données pona kotia na transaction.
    • Logging ekendaka na se ya oyo — Na kotalela logger oyo tosalelaka, bibliothèque ekoki kotinda ba journals tango tozali na transaction, to soki tozali kosalela datadog, ekozala na processus mosusu ya kobomba ba journals, kasi tofutaka kaka prix ya kobomba ba journals na mémoire tii tango mosala ya batch ekobomba yango na fichier.
    • Misala ya disque — Kokɔtisa CSV mpo na kokɔtisa na tableau to kobimisa tableau na CSV.
    • Misala ya kilo ya CPU — Multiplication ya matrice to transformation nionso ya maths/données ya kilo eko bloquer CPU na ba opérations nionso misusu — Global Interpreter Lock ya Python e forcer thread nionso esalela CPU moko na moko, pe Django ezali unique-processé (opération nionso na transaction esalemaka en série).

   def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) for file_data in user_files: # transaction waits for this upload and so do all other connections that need access to table/rows the transaction # uses url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url) Account.objects.create(user=user, balance=0) # rest of user creation proccess

  • Ba alternatives ya kosala
    • Bozua ba données prêt avant transaction — Charge ba données ya disque/réseau avant. Soki ba données esili, sala transaction na yango.
    • Salelá ba données ya placeholder — Salá ensemble ya ba données ya placeholder oyo emonani (moyen te ya kosala libunga na yango mpo na ba données ya production) oyo ekoki kozwama na pete mpe ekoki koyeba. Charger yango avant transaction pe salela yango.
    • Kosala ba données — Soki ezali na ba contraintes ya uniqueté na ba champs oyo epekisaka ba données ya placeholder esalelamaka.
    • Yango ezali na likama — Ekomema na ba échecs imprévisibles mpo na nature ya ba algorithmes pseudo-aléatoires — yango wana salela algorithme aléatoire ya malamu na mboto ya malamu mpo na koboya ba échecs ya kokamwa.
    • Longola to kobongisa ba contraintes — Soki ba contraintes ezali kopesa biso nzela te ya kosala ba enregistrements na biso sans danger, problème ezali na schéma na biso. Kosala ete ba contraintes ezala dépendante na colonne ya état oyo elakisaka tango nini enregistrement esili mpe esengeli kolandela yango.

 # not bad def transaction(user_data, user_files): user = None with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess for file_data in user_files: url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url) # best fix from functools import partial def transaction(user_data, user_files): user = None with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess # partials create a callable with the function and arguments # so that the function is called with the arguments when the transaction is committed # TODO: diff between partial and lambda here??? transaction.on_commit(partial(create_user_files, user_files, user)) def create_user_files(user_files, user): for file_data in user_files: url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url)

Next — Ndenge nini ba commandes ya PSQL e bloquer moko na mosusu

Na post oyo ekolanda, toko plonge na PSQL pe tokoyeba:

• Ndenge nini ba commandes différentes e bloquer ba tableaux na ba lignes
• Ba commandes nini ezali na risque mingi pona ko exécuter

Maziba oyo euti na yango

• Atomique
• Migrations & Transactions ya Django
  • Na ba bases de données oyo esungaka ba transactions ya DDL (SQLite na PostgreSQL), ba opérations nionso ya migration ekotambola na kati ya transaction moko par défaut. Na bokeseni, soki base de données esimbaka ba transactions ya DDL te (par exemple MySQL, Oracle) alors ba opérations nionso ekotambola sans transaction.
  • Okoki kopekisa migration esala na transaction na kotiya attribut atomique na Lokuta. Na ndakisa:
  • Ezali mpe na nzela ya kosala biteni ya migration na kati ya transaction na kosalelaka atomic() to na koleka atomic=True na RunPython
• Partitions ya PSQL & Django
  • ORM ya Django ezali na soutien intégré te mpo na ba tableaux partitionnés, yango wana soki olingi kosalela ba partitions na application na yo, ekozua mua mosala ya likolo.
  • Lolenge moko ya kosalela ba partitions ezali ya ko rouler ba migrations na yo moko oyo ezo tambuisa SQL brut. Yango ekosala, kasi elakisi ete okozala na posa ya ko gérer manuellement ba migrations pona ba changements nionso okosala na tableau na mikolo ekoya.
  • Option mosusu ezali kosalela paquet oyo babengi django-postgres-extra. Django-postgres-extra epesaka lisungi mpo na makambo mingi ya PostgreSQL oyo etongami te na ORM ya Django, ndakisa, lisungi mpo na TRUNCATE TABLE mpe bokaboli ya tableau.
• Ba Transactions ya PSQL
• Python: Ndenge nini ba transactions ya Django esalaka