Insufficient memory

Solved
Uryon Posted messages 77 Status Member -  
Uryon Posted messages 77 Status Member -
Hello everyone,

I am currently an intern and I need to create a duplicate management module for my company's database. I am working with Access 2007 and I have made good progress on the project so far (I am adding duplicates from the table Nomtable to a table called Nomtable_doublon, then I delete the duplicates from NomTable before re-adding all the unique duplicates back into Nomtable from Nomtable_doublon after processing).
The program works fine until it gives me an insufficient memory error; the database may be corrupted...
From what I have gathered online, this could be due to exceeding 2GB (even though I compact the database before any processing), I tried splitting the database but after checking, the size of the database containing the forms has not decreased.
Do you have any idea how to resolve my memory issue? Am I processing incorrectly? Could I increase the maximum memory?

Thank you in advance.
Uryon.

Configuration: Windows 7 / Chrome 26.0.1410.64

6 answers

  1. Sugel Posted messages 4293 Registration date   Status Member Last intervention   728
     
    0
    1. Sugel Posted messages 4293 Registration date   Status Member Last intervention   728
       
      Others suggest creating a new database by importing all the content from the old one.
      0
  2. Uryon Posted messages 77 Status Member
     
    I did not explain well, I can start Access without any issues, the problem occurs when I run the program (after a certain processing time, I concluded that it was when the file exceeds 2GB).
    0
    1. Sugel Posted messages 4293 Registration date   Status Member Last intervention   728
       
      Je suis désolé, mais je ne peux pas vous aider avec ça.
      0
    2. Uryon Posted messages 77 Status Member
       
      This is the solution I am currently experimenting with after finding a post on this forum, but I admit that it's not very clean and it poses a problem with the opening.
      Unless I'm mistaken, every time Access executes a program related to a linked table, it opens and closes the linked database, which adds processing time I believe, and given that the processing is already relatively long, I would like a better alternative.
      0
  3. Tessel75
     
    Hello,
    Tutorials usually say that Access has a maximum capacity of 1GB or even 750MB. The only option left is to fragment, for example, the tables on one, and the queries, forms, reports, etc. on the other.
    Good luck!
    0
  4. Uryon Posted messages 77 Status Member
     
    Currently, I have split the database into two: one contains the base tables and the other contains all the tables, queries, and forms related to processing. We'll see if it works correctly if you tell me there are no other solutions.
    Thank you.
    0
  5. blux Posted messages 2049 Registration date   Status Moderator Last intervention   3 455
     
    Hi,

    do you make your modifications using standard queries or VBA queries with/without logging (commit/rollback)?

    --
    See you later, blux
     "Stupid people dare to do anything. That's how we recognize them"
    0
  6. Uryon Posted messages 77 Status Member
     
    Sorry, I can only assist you with translations. How can I help you with that?
    0
    1. blux Posted messages 2049 Registration date   Status Moderator Last intervention   3 455
       
      It is therefore likely that multiple updates cause an inflation of database management pointers.
      It would be interesting to make some 'commits' to see if the behavior is the same...
      0
    2. Uryon Posted messages 77 Status Member
       
      I looked around a bit on the internet and found that "commits" are related to transactions, but I don't know what that is and I couldn't find...
      From what I've understood, the "commit" is used to make the desired changes to the records, is that right?
      0
    3. blux Posted messages 2049 Registration date   Status Moderator Last intervention   3 455
       
      The commit ensures consistency in multiple updates. It's the foundation of an ACID transaction.
      I would make a little commit, but before that, I would reset the recordset objects to zero from time to time.
      It would still be interesting to see your code to get an idea of what can be done to avoid this problem.
      0
    4. Uryon Posted messages 77 Status Member
       
      I will put the code just after this message.
      I managed to find a site explaining the usefulness of transactions.
      The problem is that here, based on certain criteria (entry date, highest ID, field value) I need to modify certain fields (if a field is empty, null, or equal to 0, we replace it with another to avoid data loss) so I don't think I can use transactions in this case because they wouldn't allow me to modify only part of the row, right?
      0
    5. Uryon Posted messages 77 Status Member
       
       'Allows to remove duplicates based on telephone_standard
      Private Sub dbl_telephone_Click()
          Dim db As Database
          Set db = CurrentDb()
          Dim idPrec As String 'ID of the company to keep
          Dim IdActu As String 'ID of the company to remove
          Dim sql As String
          Dim rs As Recordset
          Dim rsPrec As Recordset
          Dim rsActu As Recordset
          Dim rsPostal As Recordset
          Dim rsSaisiePrec As Recordset
          Dim rsSaisieActu As Recordset
          Dim compteur As Long
          Dim modif As Boolean
          Dim i As Integer
          Dim signaprec As Integer
      
          DoCmd.SetWarnings False 'Allows to suppress warnings during modification, addition, deletion...
          DoCmd.RunSQL ("Delete * from doublonagarder;")
          DoCmd.RunSQL ("Delete * from societe_doublon;")
      
          'Insert into societe_doublon the duplicates based on telephone_standard
          DoCmd.RunSQL ("Insert INTO societe_doublon select * FROM societe WHERE (((societe.telephone_standard) In (SELECT [telephone_standard] FROM [societe] As Tmp GROUP BY [telephone_standard] HAVING Count(*)>1 ))) order by telephone_standard, id_societe DESC;")
      
          'Initialization
          compteur = 0
          modif = False
          Set rs = db.OpenRecordset("Select * FROM societe_doublon order by telephone_standard, id_societe DESC")
      
          If rs.EOF = True Then
              MsgBox ("There are no duplicate companies ")
          Else
              rs.MoveFirst
              idPrec = rs!id_societe 'Assigning the ID of the first record to idPrec
              rs.MoveNext
      
              'Looping through the entire database
              Do While rs.EOF = False
                  IdActu = rs!id_societe
                  sql = "Select * FROM societe_doublon WHERE id_societe=" & idPrec & ";"
                  Set rsPrec = db.OpenRecordset(sql) 'Row that we will keep (idPrec)
      
                  sql = "Select * FROM societe_doublon WHERE id_societe=" & IdActu & ";"
                  Set rsActu = db.OpenRecordset(sql) 'Row that we will remove (idActu)
      
                  signaprec = rsPrec!signaletique_ok 'If the telephone_standard is identical then
                  If rsPrec!telephone_standard = rsActu!telephone_standard Then
                      'STEP 1: RsPrec is the row we will keep, if any of its fields are empty, we replace them with those from RsActu
                      rsPrec.Edit
                      For i = 1 To rsPrec.Fields.Count - 3
                          If IsNull(rsPrec.Fields(i).Value) = True Or rsPrec.Fields(i).Value = "" Or rsPrec.Fields(i).Value = 0 Then
                              rsPrec.Fields(i).Value = rsActu.Fields(i).Value
                          End If
                      Next i
                      rsPrec!signaletique_ok = signaprec
                      rsPrec.Update 'END OF STEP 1
      
                      'STEP 2: Checking the signaletique_ok
                      'We check if signaletique_ok of rsActu equals 1, if yes and that of rsPrec is 0 then we replace the fields of rsPrec with those of rsActu
                      If rsPrec!signaletique_ok = 0 And rsActu!signaletique_ok = 1 Then
                          modif = True
                      End If
      
                      'If signaletique_ok of rsPrec and rsActu have the same value
                      If (rsPrec!signaletique_ok = 1 And rsActu!signaletique_ok = 1) Or (rsPrec!signaletique_ok = 0 And rsActu!signaletique_ok = 0) Then
                          Set rsSaisieActu = db.OpenRecordset("Select date_saisie FROM actions WHERE id_societe=" & IdActu & " ORDER BY date_saisie DESC;") 'Date of entry of rsActu
                          Set rsSaisiePrec = db.OpenRecordset("Select date_saisie FROM actions WHERE id_societe=" & idPrec & " ORDER BY Date_saisie DESC;") 'Date of entry of rsPrec
      
                          'Check that rsPrec and rsActu have an entry date
                          If rsSaisiePrec.EOF = False And rsSaisieActu.EOF = False Then
                              rsSaisiePrec.MoveFirst
                              rsSaisieActu.MoveFirst
                              'If the entry date of RsActu is more recent than that of rsPrec then we replace the fields of rsPrec with those of rsActu
                              If rsSaisiePrec!date_saisie < rsSaisieActu!date_saisie Then
                                  modif = True
                              End If
                          End If
                          rsSaisieActu.Close
                          rsSaisiePrec.Close
                      End If 'END OF STEP 2
      
                      'STEP 3: We modify the fields of rsPrec with those of rsActu if necessary
                      If modif = True Then
                          rsPrec.Edit
                          For i = 1 To rsPrec.Fields.Count - 3
                              If IsNull(rsActu.Fields(i).Value) = False And rsActu.Fields(i).Value <> "" And rsPrec.Fields(i).Value <> 1 Then
                                  rsPrec.Fields(i).Value = rsActu.Fields(i).Value
                              End If
                          Next i
                          rsPrec.Update
                      End If 'END OF STEP 3
      
                      'STEP 4: We check that if the country is France, the postal code matches the first two digits of the telephone_standard
                      If rsActu!pays = "France" Then
                          sql = "Select * FROM TablePostal WHERE code_Postal='" & Left(rsPrec!code_postal, 2) & "';"
                          Set rsPostal = db.OpenRecordset(sql) 'Row containing the telephone index for the postal code of rsPrec
                          If rsPostal.EOF = False Then
                              'If the telephone index and the postal code do not match then we replace the postal code of rsPrec with that of rsActu
                              If Left(rsPrec!tel2, 2) <> rsPostal!indice_tel Then
                                  rsPrec.Edit
                                  rsPrec!code_postal = rsActu!code_postal
                                  rsPrec!ville = rsActu!ville
                                  rsPrec.Update
                                  rsPostal.Close
                              End If
                          Else
                              'If the postal code of rsPrec does not match any telephone index then we replace the postal code of rsPrec with that of rsActu
                              rsPrec.Edit
                              rsPrec!code_postal = rsActu!code_postal
                              rsPrec!ville = rsActu!ville
                              rsPrec.Update
                              rsPostal.Close
                          End If
                      End If 'END OF STEP 4
      
                      rsPrec.Close
                      rsActu.Close
                      DoCmd.RunSQL ("Update societe_doublon SET doublon=1 WHERE id_societe=" & IdActu & ";")
                      DoCmd.RunSQL ("Update societe_doublon SET doublon=2 WHERE id_societe=" & idPrec & ";")
                      compteur = compteur + 1
                      DoCmd.RunSQL ("Insert INTO TableCorrespondance (ID_ancienne,ID_nouvelle) values (" & IdActu & "," & idPrec & ");")
                  Else 'If it is not a duplicate then we assign idprec the value of idactu
                      rsPrec.Close
                      rsActu.Close
                      idPrec = IdActu
                  End If
                  rs.MoveNext
              Loop
          End If
          rs.Close
      
          DoCmd.RunSQL ("Insert INTO DoublonAjeter Select * FROM societe_doublon WHERE doublon=1 AND id_societe NOT IN (Select ID_societe FROM doublonajeter);")
          DoCmd.RunSQL ("Insert INTO DoublonAgarder Select * FROM societe_Doublon WHERE doublon=2 AND id_societe NOT IN (select id_societe FROM doublonagarder);")
          DoCmd.RunSQL ("Delete * from societe WHERE id_societe IN (Select id_societe FROM societe_doublon);")
          DoCmd.RunSQL ("Insert INTO societe select * from doublonagarder;")
      
          MsgBox ("There were " & compteur & " duplicates removed")
      End Sub
      0