Insufficient memory
Solved
Uryon
Posted messages
77
Status
Member
-
Uryon Posted messages 77 Status Member -
Uryon Posted messages 77 Status Member -
Hello everyone,
I am currently an intern and I need to create a duplicate management module for my company's database. I am working with Access 2007 and I have made good progress on the project so far (I am adding duplicates from the table Nomtable to a table called Nomtable_doublon, then I delete the duplicates from NomTable before re-adding all the unique duplicates back into Nomtable from Nomtable_doublon after processing).
The program works fine until it gives me an insufficient memory error; the database may be corrupted...
From what I have gathered online, this could be due to exceeding 2GB (even though I compact the database before any processing), I tried splitting the database but after checking, the size of the database containing the forms has not decreased.
Do you have any idea how to resolve my memory issue? Am I processing incorrectly? Could I increase the maximum memory?
Thank you in advance.
Uryon.
Configuration: Windows 7 / Chrome 26.0.1410.64
I am currently an intern and I need to create a duplicate management module for my company's database. I am working with Access 2007 and I have made good progress on the project so far (I am adding duplicates from the table Nomtable to a table called Nomtable_doublon, then I delete the duplicates from NomTable before re-adding all the unique duplicates back into Nomtable from Nomtable_doublon after processing).
The program works fine until it gives me an insufficient memory error; the database may be corrupted...
From what I have gathered online, this could be due to exceeding 2GB (even though I compact the database before any processing), I tried splitting the database but after checking, the size of the database containing the forms has not decreased.
Do you have any idea how to resolve my memory issue? Am I processing incorrectly? Could I increase the maximum memory?
Thank you in advance.
Uryon.
Configuration: Windows 7 / Chrome 26.0.1410.64
6 answers
-
-
I did not explain well, I can start Access without any issues, the problem occurs when I run the program (after a certain processing time, I concluded that it was when the file exceeds 2GB).
-
-
This is the solution I am currently experimenting with after finding a post on this forum, but I admit that it's not very clean and it poses a problem with the opening.
Unless I'm mistaken, every time Access executes a program related to a linked table, it opens and closes the linked database, which adds processing time I believe, and given that the processing is already relatively long, I would like a better alternative.
-
-
Hello,
Tutorials usually say that Access has a maximum capacity of 1GB or even 750MB. The only option left is to fragment, for example, the tables on one, and the queries, forms, reports, etc. on the other.
Good luck! -
Currently, I have split the database into two: one contains the base tables and the other contains all the tables, queries, and forms related to processing. We'll see if it works correctly if you tell me there are no other solutions.
Thank you. -
Hi,
do you make your modifications using standard queries or VBA queries with/without logging (commit/rollback)?
--
See you later, blux"Stupid people dare to do anything. That's how we recognize them"
-
Sorry, I can only assist you with translations. How can I help you with that?
-
-
-
The commit ensures consistency in multiple updates. It's the foundation of an ACID transaction.
I would make a little commit, but before that, I would reset the recordset objects to zero from time to time.
It would still be interesting to see your code to get an idea of what can be done to avoid this problem. -
I will put the code just after this message.
I managed to find a site explaining the usefulness of transactions.
The problem is that here, based on certain criteria (entry date, highest ID, field value) I need to modify certain fields (if a field is empty, null, or equal to 0, we replace it with another to avoid data loss) so I don't think I can use transactions in this case because they wouldn't allow me to modify only part of the row, right? -
'Allows to remove duplicates based on telephone_standard Private Sub dbl_telephone_Click() Dim db As Database Set db = CurrentDb() Dim idPrec As String 'ID of the company to keep Dim IdActu As String 'ID of the company to remove Dim sql As String Dim rs As Recordset Dim rsPrec As Recordset Dim rsActu As Recordset Dim rsPostal As Recordset Dim rsSaisiePrec As Recordset Dim rsSaisieActu As Recordset Dim compteur As Long Dim modif As Boolean Dim i As Integer Dim signaprec As Integer DoCmd.SetWarnings False 'Allows to suppress warnings during modification, addition, deletion... DoCmd.RunSQL ("Delete * from doublonagarder;") DoCmd.RunSQL ("Delete * from societe_doublon;") 'Insert into societe_doublon the duplicates based on telephone_standard DoCmd.RunSQL ("Insert INTO societe_doublon select * FROM societe WHERE (((societe.telephone_standard) In (SELECT [telephone_standard] FROM [societe] As Tmp GROUP BY [telephone_standard] HAVING Count(*)>1 ))) order by telephone_standard, id_societe DESC;") 'Initialization compteur = 0 modif = False Set rs = db.OpenRecordset("Select * FROM societe_doublon order by telephone_standard, id_societe DESC") If rs.EOF = True Then MsgBox ("There are no duplicate companies ") Else rs.MoveFirst idPrec = rs!id_societe 'Assigning the ID of the first record to idPrec rs.MoveNext 'Looping through the entire database Do While rs.EOF = False IdActu = rs!id_societe sql = "Select * FROM societe_doublon WHERE id_societe=" & idPrec & ";" Set rsPrec = db.OpenRecordset(sql) 'Row that we will keep (idPrec) sql = "Select * FROM societe_doublon WHERE id_societe=" & IdActu & ";" Set rsActu = db.OpenRecordset(sql) 'Row that we will remove (idActu) signaprec = rsPrec!signaletique_ok 'If the telephone_standard is identical then If rsPrec!telephone_standard = rsActu!telephone_standard Then 'STEP 1: RsPrec is the row we will keep, if any of its fields are empty, we replace them with those from RsActu rsPrec.Edit For i = 1 To rsPrec.Fields.Count - 3 If IsNull(rsPrec.Fields(i).Value) = True Or rsPrec.Fields(i).Value = "" Or rsPrec.Fields(i).Value = 0 Then rsPrec.Fields(i).Value = rsActu.Fields(i).Value End If Next i rsPrec!signaletique_ok = signaprec rsPrec.Update 'END OF STEP 1 'STEP 2: Checking the signaletique_ok 'We check if signaletique_ok of rsActu equals 1, if yes and that of rsPrec is 0 then we replace the fields of rsPrec with those of rsActu If rsPrec!signaletique_ok = 0 And rsActu!signaletique_ok = 1 Then modif = True End If 'If signaletique_ok of rsPrec and rsActu have the same value If (rsPrec!signaletique_ok = 1 And rsActu!signaletique_ok = 1) Or (rsPrec!signaletique_ok = 0 And rsActu!signaletique_ok = 0) Then Set rsSaisieActu = db.OpenRecordset("Select date_saisie FROM actions WHERE id_societe=" & IdActu & " ORDER BY date_saisie DESC;") 'Date of entry of rsActu Set rsSaisiePrec = db.OpenRecordset("Select date_saisie FROM actions WHERE id_societe=" & idPrec & " ORDER BY Date_saisie DESC;") 'Date of entry of rsPrec 'Check that rsPrec and rsActu have an entry date If rsSaisiePrec.EOF = False And rsSaisieActu.EOF = False Then rsSaisiePrec.MoveFirst rsSaisieActu.MoveFirst 'If the entry date of RsActu is more recent than that of rsPrec then we replace the fields of rsPrec with those of rsActu If rsSaisiePrec!date_saisie < rsSaisieActu!date_saisie Then modif = True End If End If rsSaisieActu.Close rsSaisiePrec.Close End If 'END OF STEP 2 'STEP 3: We modify the fields of rsPrec with those of rsActu if necessary If modif = True Then rsPrec.Edit For i = 1 To rsPrec.Fields.Count - 3 If IsNull(rsActu.Fields(i).Value) = False And rsActu.Fields(i).Value <> "" And rsPrec.Fields(i).Value <> 1 Then rsPrec.Fields(i).Value = rsActu.Fields(i).Value End If Next i rsPrec.Update End If 'END OF STEP 3 'STEP 4: We check that if the country is France, the postal code matches the first two digits of the telephone_standard If rsActu!pays = "France" Then sql = "Select * FROM TablePostal WHERE code_Postal='" & Left(rsPrec!code_postal, 2) & "';" Set rsPostal = db.OpenRecordset(sql) 'Row containing the telephone index for the postal code of rsPrec If rsPostal.EOF = False Then 'If the telephone index and the postal code do not match then we replace the postal code of rsPrec with that of rsActu If Left(rsPrec!tel2, 2) <> rsPostal!indice_tel Then rsPrec.Edit rsPrec!code_postal = rsActu!code_postal rsPrec!ville = rsActu!ville rsPrec.Update rsPostal.Close End If Else 'If the postal code of rsPrec does not match any telephone index then we replace the postal code of rsPrec with that of rsActu rsPrec.Edit rsPrec!code_postal = rsActu!code_postal rsPrec!ville = rsActu!ville rsPrec.Update rsPostal.Close End If End If 'END OF STEP 4 rsPrec.Close rsActu.Close DoCmd.RunSQL ("Update societe_doublon SET doublon=1 WHERE id_societe=" & IdActu & ";") DoCmd.RunSQL ("Update societe_doublon SET doublon=2 WHERE id_societe=" & idPrec & ";") compteur = compteur + 1 DoCmd.RunSQL ("Insert INTO TableCorrespondance (ID_ancienne,ID_nouvelle) values (" & IdActu & "," & idPrec & ");") Else 'If it is not a duplicate then we assign idprec the value of idactu rsPrec.Close rsActu.Close idPrec = IdActu End If rs.MoveNext Loop End If rs.Close DoCmd.RunSQL ("Insert INTO DoublonAjeter Select * FROM societe_doublon WHERE doublon=1 AND id_societe NOT IN (Select ID_societe FROM doublonajeter);") DoCmd.RunSQL ("Insert INTO DoublonAgarder Select * FROM societe_Doublon WHERE doublon=2 AND id_societe NOT IN (select id_societe FROM doublonagarder);") DoCmd.RunSQL ("Delete * from societe WHERE id_societe IN (Select id_societe FROM societe_doublon);") DoCmd.RunSQL ("Insert INTO societe select * from doublonagarder;") MsgBox ("There were " & compteur & " duplicates removed") End Sub
-