Author Topic: [MOD] AutoResearch Version 1.0  (Read 28090 times)

Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 05, 2018, 12:49:42 AM
Yeah, until you need to / want to update, and you basically have to reinstall everything from scratch to ensure your mods don't overwrite updated game assets. There's a lot of room for improvement on this part of the modding experience.

That aside - not a bad idea. I guess that's kind of what I do already: I have a separate folder somewhere with all the ZIP files of the mods I have installed.
Shrooblord



wm46

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    16
    • View Profile
on: February 05, 2018, 07:28:22 AM
Your fix isn't working for me Shroob. I don't know if you had noticed this, but in the research screen all items are doubled. Not actually doubled, but indexed twice. When I hit the research button, none of my turrets get researched.

I assume what's happening is that because of the double indexing, the auto research function is putting in the same item twice along with one other item. Let's say the last 3 item indicies for my inventory are [33] [34] [35], they show up in the research menu as [33] [33] [34] [34] [35] [35], and I'm assuming it tries to combine "[35] [35] [34] -> not enough items to combine"



Don't mind the blank icon for the legendary weapon, it was generated with GenerateTurretTemplate(), but with a DPS value of 10k+ to give a size ~20 turret.
« Last Edit: February 05, 2018, 07:35:36 AM by wm46 »



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 05, 2018, 05:41:20 PM
Meh, sorry about that. I must admit I pumped out an "update" in a hurry. You can also tell by the amount of edits I made to that post detailing some bugs I found right afterwards.

My experiences with the "fix" so far are:
  • It works if you have a lot of System Module Upgrades or Turrets to research.
  • It breaks non-Auto Research (vanilla) functionality.
  • It doesn't combine turrets of different types like players might; it only Auto-Researches duplicates (i.e. those made in a Turret Factory).

On top of what you just added, I'd say the "fix" is rather buggy and shouldn't be installed, correct. x)

I should take a look at this again in the future, but I remember having quite the headscratcher trying to resolve the original bugs I found. Perhaps it would be best to try and rewrite this entire thing from scratch.

---

That said, I did have a little bit of a revelation which might work: instead of using the item.uvalue variables I mentioned giving me trouble before, one could use entity:getValue(...) and entity:setValue(...) calls to achieve the same effect. It may be worth a shot.

To be continued...
Shrooblord



Gnarshke

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    1
    • View Profile
on: February 13, 2018, 08:31:13 PM
Hey mate,

still not working for me. Full Inventory but nothing to research ... *strange*




raz334

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    21
    • View Profile
on: February 14, 2018, 12:31:20 AM
Here, I've fixed the items not being found issue so the Auto Research option will work now, sometimes, usually at least the first time per restart of game.  It was just a simple math error in the previous code.

Unfortunately the manual research still does not work, if anyone could post the original games Research station file, i might be able to figure out whats wrong with the Auto-research version but im not going to wipe what i have just to do that.. Enjoy



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 14, 2018, 07:19:29 PM
Thanks for your efforts!

im not going to wipe what i have just to do that..
You needn't, necessarily: rename your /data/ folder to /dataBKUP/ and run Steam Integrity Verification. You'll get a new /data/ folder with fresh vanilla files. Now rename that new folder to /dataVanilla/ and your /dataBKUP/ back to /data/. You now have both your modded files, as well as a nice store of vanilla files to plunder. ;)
Shrooblord



raz334

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    21
    • View Profile
on: February 14, 2018, 10:13:38 PM
Thanks i should have thought of that !

Also glad to see your not upset about my getting involved with fixing it, not sure if you've used this site before but check this:  https://www.diffchecker.com/h7d1BTUD

It's comparing the vanilla file and the modded file, so that all the differences are easily readable.  unfortunately i am not very skilled with lua languages im more of a java programmer, but i'll do what i can to assist you.



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 14, 2018, 10:42:40 PM
Thanks i should have thought of that !
Actually it's a good tip for both of us to get our hands on some vanilla files while keeping the game modded. Should've thought of it sooner. xD

Quote
Also glad to see your not upset about my getting involved with fixing it
Not at all, not at all! I love people working together on something. I'm quite busy with my own mods and a mod manager, myself, so I gave up on this project for a bit until I had inspiration again to start hacking at it. More power to the community for picking up after my slacking ass! ;d

Quote
not sure if you've used this site before but check this:  https://www.diffchecker.com/h7d1BTUD
Haven't I just! I love DiffChecker. It's so nice. Bar an actual git history, it's the best thing for comparing people's work efforts against vanilla files or other people's work. Top notch!

Quote
It's comparing the vanilla file and the modded file, so that all the differences are easily readable.  unfortunately i am not very skilled with lua languages im more of a java programmer, but i'll do what i can to assist you.
Yes, and I'm very curious: what exactly did you change to get it to work? I would love to learn more about this (and I'm currently engaged in other projects as I mentioned before, so a little explanation would do wonders for my mind rather than having to dive into it myself and start researching it again - if you please).

I'm specifically talking about this part of the problem:
EDIT II:
Oh shit, looks like I broke the regular Research by introducing this change. I'll try and fix that bug. One sec.

EDIT III:
I can't seem to work it out. The problem is in this part of the code. researchstation.lua, line 451 is the conflict. It claims that item.uvalue is nil, therefore the array index doesn't work. As a result, it can't execute the function. Meaning: you can't use the manual Research method with this mod installed. Lame.
Code: [Select]
for _, item in pairs(required:getItems()) do
        table.insert(items, item.item)

        local amount = itemIndices[item.uvalue] or 0
        amount = amount + 1
        itemIndices[item.uvalue] = amount
    end
    for _, item in pairs(optional:getItems()) do
        table.insert(items, item.item)

        local amount = itemIndices[item.uvalue] or 0
        amount = amount + 1
        itemIndices[item.uvalue] = amount
    end

I'll take another crack at it tomorrow, but for now, I'd love to ask all of you to pour your wisdom into this while I catch some shut-eye. ^^'
How did you get it to accept the item.uvalue changes?

EDIT:
Wait, is it the fact that you changed amount = amount + 1 to amount = amount? I wonder, how does this work and what does it break by doing this? The amount variable was being incremented for a reason, I assume? Now I'm even more curious as to how you fixed it. ;D
« Last Edit: February 14, 2018, 10:46:16 PM by Shrooblord »
Shrooblord



raz334

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    21
    • View Profile
on: February 15, 2018, 03:45:58 AM
The original vanilla file was using an index type of variable for the amount, im not sure if your aware of this, i would assume so. Anyway an index variable always starts at zero, which is why the vanilla files had to add a plus one to the increments. Now i am not familiar with the uvalue command as it is not in java, however after testing it i believe it is safe to assume that the increment is not needed for it.

Also i only got the inventory and autoresearch fixed not the manual version like in vanilla...
I have however been working on this and i believe i may have an idea as to the problem, which i may need your help resolving.  It appears that this section in the autoresearch.lua is not calling back to the onClickResearch() command like it should...

function onClickAutoResearch()
if not cbAutoResearch.checked then onClickResearch()
return

ps. you really should use returns and indents in your code. it was quite the headache to find this.  Including a slightly cleaned version of the file for your sake.

Included file is useless to users...

**Changed Posted File to better reflect clean code, only in the beginning though**
« Last Edit: February 15, 2018, 04:06:13 AM by raz334 »



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 15, 2018, 04:30:12 AM
ps. you really should use returns and indents in your code. it was quite the headache to find this.  Including a slightly cleaned version of the file for your sake.
Ahaha, indeed, indeed. But if you would see code that I write, you would know that I make it as legible as possible, complete with comments, mostly for my own sake. Here, have a snippet from something I'm working on right now:

Code: [Select]
--By coord (first by X, asc/des, then by Y, asc/des)
function InactiveGate.onCoordLabelClick()
    --X sorting
    if sortByCoord < 2 then
        InactiveGate.setSortFunction(InactiveGate.sortByXCoordAsc, InactiveGate.sortByXCoordDes)
        sortByCoord = sortByCoord + 1
    --Y sorting
    else
        InactiveGate.setSortFunction(InactiveGate.sortByYCoordDes, InactiveGate.sortByYCoordAsc)    --Des first instead of Asc because of how the galaxy coordinates work
        sortByCoord = sortByCoord + 1
        --reset to X sorting
        if sortByCoord >= 4 then
            sortByCoord = 0
        end
    end
end

(...)

-- client-sided
function InactiveGate.updateGates()
    if inactiveGatesLBE == nil then return end
    --if gateData == nil then return end
    inactiveGatesLBE:clear()
   
    table.sort(gateData.gates, sortFunction)
   
    for i, pair in pairs(gateData.gates) do 
       
        local coords = InactiveGate.padCoords(pair.x, pair.y)
       
        local timeTC = InactiveGate.formatTime(pair.time)
       
        local money = "$"..createMonetaryString(round(pair.cost))
           
        inactiveGatesLBE:addRow( pair.name%_t, coords, tRN(pair.dist), money, timeTC )
    end
end

You'll see the coding style is much different from this mod's original author's. While I am flattered that you seem to suggest I wrote this mod (at least, that's how I read into it - maybe I'm making a fool out of myself right now, but let's say I'm right), I must inform you out of decency that this mod is not my work but Cypher's.

I highly doubt that Cypher coded this as it was in its minified state. If they did, then they're even more of a magician than I already hold them for.

Good work making the code more easily parsed by human eyes. I've done the same when trying to decipher what it was doing. To be honest, though, the multiple single-line lua operations this code was cram-packed full of has opened my eyes to a new sort of black magicks, and I find that wizardry quite profound.

====

That said, let's talk about bugfixing the issues this mod has in its current state.

The original vanilla file was using an index type of variable for the amount (...). An index variable always starts at zero, which is why the vanilla files had to add a plus one to the increments.
I'm not sure I understand. How does an index always remain zero? Do you mean it's zero-based, i.e. the first element (index) is "element 0", like with most programming languages' arrays (eg. name[0] = "Fred")?

If not, I will ask again: what use is an indexing variable if it always remains the same value (regardless of that being 0, 15 or "Millie Bobby Brown")? I'll put the old vanilla code and your changes side-by-side:
Code: [Select]
416. for _, item in pairs(required:getItems()) do                 446.     for _, item in pairs(required:getItems()) do
417.         table.insert(items, item.item)                         447.         table.insert(items, item.item)
418.                                                         448.
419.         local amount = itemIndices[item.index] or 0             449.         local amount = itemIndices[item.uvalue] or 0
420.         amount = amount + 1                                 450.         amount = amount
421.         itemIndices[item.index] = amount                 451.         itemIndices[item.uvalue] = amount
422. end                                                                452.        end
See, in the original, amount is keeping track of something. For every iteration of the for loop, the amount gets incremented. However, with your changes, amount is no longer changed, ever. It stays the same. In fact, it gets set to its current value (which is in fact a little pointless, but I digress).

I'm not trying to reprimand you here or anything - just trying to understand how this change actually solved the buggy behaviour. I'm curious.

Quote
Now i am not familiar with the uvalue command as it is not in java, however after testing it i believe it is safe to assume that the increment is not needed for it.
And this is why I'm curious. I wonder what amount was in charge of keeping track of, if completely taking it out of the equation solves the bug. I also wonder what other, yet unspotted bugs it may have caused. But maybe there are no new bugs. Let's hope, until we do some more testing.

Quote
Also i only got the inventory and autoresearch fixed not the manual version like in vanilla...
Yes, too bad. That's also the snag I hit at some point. I was hoping it would be a simple on/off switch type of deal, but the problem seems more deep-rooted than that.

Quote
I have however been working on this and i believe i may have an idea as to the problem, which i may need your help resolving.  It appears that this section in the autoresearch.lua is not calling back to the onClickResearch() command like it should...

function onClickAutoResearch()
if not cbAutoResearch.checked then onClickResearch()
return
Good find!

In my own tests, I found out I had to make several adjustments to the autoresearch.lua script in order to make it compliant with the current version of the game. One of such changes was adding a namespace, implementing proper returns (as you say), and properly calling the functions by their new names.

onClickResearch() is not a function that exists in the namespace of autoresearch.lua. That is to say, the script autoresearch.lua doesn't have a function onClickResearch() defined anywhere inside of it. This makes sense, as the original mod's code appears to want to call functions that are inside the vanilla file researchstation.lua. However, it doesn't, because this has not been properly set up (because the way that the code is managed in the current version of Avorion is different to the method employed back when this mod was first created).

The debugger in Avorion is really good. It gives precise errors and stacktraces, and manages to catch exceptions like a beast. One problem, however, if I recall correctly, is that function calls that reference non-existent functions, silently fail. Now don't quote me on that: it's been a while since I've tested that, but it could be that that's why we don't catch this behaviour going wrong when autoresearch.lua tries to execute: the debugger (printing to Console) simply doesn't know about the mistake, because no error is raised.

So how to fix this? We need to properly include namespaces. I refer back to the code I posted of my own mod project:
Code: [Select]
function InactiveGate.updateGates()
See that InactiveGate. in front of the function's name? That's the namespace indicator. It's great, because with it implemented, I can call updateGates() from any script given it's executing in the same scope (Entity, Sector, Server... that kind of thing), by calling InactiveGate.updateGates() from that other script. I've done it several times before in my mod projects by now.

So try that. Give researchstation.lua a namespace (look to many, many other scripts in Avorion that already employ this) and make onClickResearch() callable from outside of the script itself by including it in the namespace. Then call it within autoresearch.lua using namespace.onClickResearch() instead of the current onClickResearch() and see if that works. You can run a simple print() command at every step of the way and check Console to see if stuff is behaving as you'd expect.

Good luck! Hope you crack this one. Looking forward to updates. :)
« Last Edit: February 15, 2018, 04:34:54 AM by Shrooblord »
Shrooblord



raz334

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    21
    • View Profile
on: February 15, 2018, 04:47:08 AM
Wow, that is a really long message, i'll do my best to answer everything..

"I'm not sure I understand. How does an index always remain zero? Do you mean it's zero-based, i.e. the first element (index) is "element 0", like with most programming languages' arrays (eg. name[0] = "Fred")?"
++++++++
     Yes it is an array index
++++++++
"See, in the original, amount is keeping track of something. For every iteration of the for loop, the amount gets incremented. However, with your changes, amount is no longer changed, ever. It stays the same. In fact, it gets set to its current value (which is in fact a little pointless, but I digress)."
+++++++++
     No, it is not staying the same, the itemIndices[  ]  is a function that is determining the reference
     number for the amount, so this function actually changes the value of amount each time it is ran.

     I have actually commented out the entire lines of amount = amount, they no longer execute in my revisions
++++++++++
"And this is why I'm curious. I wonder what amount was in charge of keeping track of, if completely taking it out of the equation solves the bug"
+++++++++++
     No, no no, don't remove it lol, It is keeping track of the number of items i believe, because when it was
     a plus one on the index, every item was being either not displayed or displayed twice over.
+++++++++++

Ouch, on the namespaces, so with java we work with packages and those are quite different. I was reading up on this namespace stuff and well im confused as hell on it !
Any chance you could provide me with the declaration code for that InactiveGate you have?  maybe if i see an example of that i might understand better.

in Java i would use the keyword package somename; and then just include that into the calling file and then reference it similar to this

function onClickAutoResearch()
            if not cbAutoResearch.checked then researchstation.onClickResearch()
            return
            end;

but seeing as package is already being used and that file is already included, plus i just tested the changes and of course it didn't work...
« Last Edit: February 15, 2018, 04:55:05 AM by raz334 »



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 15, 2018, 05:21:00 AM
No, it is not staying the same, the itemIndices[  ]  is a function that is determining the reference
     number for the amount, so this function actually changes the value of amount each time it is ran.
Lol duh. I should stop coming here at 4:30 AM and expect to make any sort of sense, much less read a single line of code correctly. xD

Quote
I have actually commented out the entire lines of amount = amount, they no longer execute in my revisions
++++++++++
"And this is why I'm curious. I wonder what amount was in charge of keeping track of, if completely taking it out of the equation solves the bug"
+++++++++++
     No, no no, don't remove it lol
This is actually what I meant with taking it out of the equation; amount = amount is a moot statement, so commenting it out (or simply deleting it outright) is what I meant. ;)

Any chance you could provide me with the declaration code for that InactiveGate you have?  maybe if i see an example of that i might understand better.
Yeah, sure. The namespaces aren't that hard to get once you've seen it a couple of times and done it yourself. Here, I'll give you an example from one of my mods in the making that communicates between four different scripts:

autoDockAI.lua
Code: [Select]
local AutoDockAI = {}

AutoDockAI.usedDock = nil
AutoDockAI.dockStage = 0
AutoDockAI.printVals = 0 --set to 0 for no debug, 1 for some debug and 2 for extensive debug logging
AutoDockAI.numDocks = 0

function AutoDockAI.printUsedDock()
    if onServer() and AutoDockAI.printVals > 0 then
        print("AutoDockAI: dock: "..tostring(AutoDockAI.usedDock))
    end
end

(...)
-- more such declarations...
(...)
return AutoDockAI

autoDock.lua
Code: [Select]
package.path = package.path .. ";data/scripts/lib/?.lua"
package.path = package.path .. ";data/scripts/entity/?.lua"
package.path = package.path .. ";data/scripts/?.lua"

require("stringutility")
require("utility")
require("faction")
local AutoDockAI = require("mods/AutoDock/scripts/entity/ai/autoDockAI")
local PlanGenerator = require("plangenerator")

(...)

function arbitraryFunction()
    if AutoDockAI.dockStage == 0 then
       return 0.2
    end
end

(...)

local docked, tractorActive, target = AutoDockAI.autoDock(ship, station)

====

And the inactiveGate.lua example I gave earlier:
Code: [Select]
package.path = package.path .. ";data/scripts/lib/?.lua"
require ("randomext")
(...)  --more requires like this
require ("player")

local Dialog = require("dialogutility")

local SectorSpecifics = require ("sectorspecifics")
(...)  --more handles like this
local ShipUtility = require ("shiputility")

-- Don't remove or alter the following comment, it tells the game the namespace this script lives in. If you remove it, the script will break.
-- namespace InactiveGate
InactiveGate = {}

(...)
-- this function gets called every time the window is shown on the client, ie. when a player presses F
function InactiveGate.onShowWindow(optionIndex, material)
    local interactingFaction = Faction(Entity(Player().craftIndex).factionIndex)
    InactiveGate.sendAllGates(Player().index)
end

====

There are actually two ways to implement namespaces. One is like the autoDockAI.lua example, and the other is like the inactiveGate.lua example.

I'll start with the easier one. In the case of inactiveGate.lua, we set a namespace at the top of the file, after importing all required scripts (making them available in the scope of our current script) and adding handles to other scripts' namespaces by defining them as variables (more on that later as I discuss the other example). Copying the behaviour of the vanilla lua files, it would appear there is also a crucial comment that's included which is part of the declaration, and is necessary. I haven't actually tested not including this comment, so I don't know how true that is.

Next, we define functions, again, by declaring them as function namespace. and then call them in the same way whenever we want to call them in-script, even within that same script. Note the  InactiveGate.sendAllGates(Player().index) call, which calls a function within the same script's namespace, yet still needs to use the same syntax as if calling a different namespace's function. It's just a syntax you'll get used to.

That's it! Namespace created. To call from a different script, simply require it and use its namespace appropriately.

----

Next up, the autoDock.lua example. As you can tell, this one depends on two different script files. One of them, autoDockAI.lua is the script which has a namespace defined, while the other does not. You will note that although autoDock.lua does not have a namespace defined, it will still work. This is important: namespaces are by no means required by the game to run scripts properly. But I have found they make life a lot easier if you want to call functions from different scripts. And they just clear up confusion and make it clear to you the instant you read something: "Ah, this function is from a different script."

Anyway, back to the point.

autoDockAI.lua has a namespace, and more importantly, notice the return at the end of the script. This script never actually executes on its own. It's almost like an object, or a library of functions and variable definitions. It doesn't do anything until autoDock.lua calls it when calling the function AutoDockAI.autoDock(ship, station). Sure, it initialises and sets some variables when autoDock.lua declares that handler way at the top of the file, but it doesn't execute anything on its own.

Of course, this is because there are no automatically triggered functions declared within autoDockAI.lua. There's no ticking functions nor updating properties. So it's a dormant script.

However, it has lots of data inside of it. It has all those lovely variables and functions declared within it, which we can access at any time from the main script autoDock.lua by calling AutoDockAI.<OPERATION> whether <OPERATION> is a variable name or a function call.

====

I hope that clears up namespaces for you. They're really not that hard once you've experimented with them a little, and they'll make coding with multiple scripts so, so much easier.

====

I have to admit I'm no expert. I've seen lua scripts include other scripts in their namespace using requires alone. Others use pcall and yet others use namespaces. I'm most comfortable with namespaces right now, so that's what I'm advising you use, not because it's de facto the best approach, but just because I know how they work, and they're relatively easy to get started with (in my opinion).

Other options may prove to work better for you. Please do tell. I'm eager to learn more, always.
« Last Edit: February 15, 2018, 05:25:36 AM by Shrooblord »
Shrooblord



raz334

  • Newbie
  • *
  • Status:
    Offline
    Posts:
    21
    • View Profile
on: February 15, 2018, 06:05:02 AM
well im stuck, so im just going to provide what i have and let you finish it...
I tried to create the namespaces but the result was that the research option vanished o.0, i've made several attemps.  on the bright side the problem is found and i also took the time to reformat all of the autoresearch.lua to be much easier to read.

autoresearch.lua:
local researchstation = require("researchstation")
function onClickAutoResearch()
if not cbAutoResearch.checked then researchstation.onClickResearch()
return

researchstation.lua:
local researchstation = {}
function researchstation.onClickResearch()
--function onClickResearch()

I should probably point out that i actually changed the amount variable back to the Array Index that vanilla uses. I did manage to test that and it does work fine with that. I honestly don't know what the original author was doing when he switched it, maybe he was unfamiliar with Arrays?
« Last Edit: February 15, 2018, 06:08:43 AM by raz334 »



Shrooblord

  • Hero Member
  • *****
  • Status:
    Offline
    Posts:
    523
    • View Profile
on: February 15, 2018, 01:06:38 PM
Awesome! Thanks for all your work. ;D

I'll give it another swing some point after I'm done with my own two mods (sorry guys, a man has his priorities ;) ). If before then anyone else is hit with inspiration, please, do try, yourself.
Shrooblord



Burnthalo

  • Jr. Member
  • **
  • Status:
    Offline
    Posts:
    60
    • View Profile
on: March 09, 2018, 09:34:14 PM
great idea. it doesn't seem to work for me tho.
the inv shows nothing at all despite my having tons of items. boxes checked or unchecked for rarity makes no difference in this. after exiting the station to try to reopen it the research option completely disappeared from the docking menu.
below are links to screen shots of it
https://prnt.sc/ip1wyy
https://prnt.sc/ip1xa1
https://prnt.sc/ip1xiz

I am using latest stable version of the game. just today I got shipsscriptloader. carrier command weapon engineering and mine corp working so would think I did a drag and drop right.. who knows.
even after uninstalling my inv doesnts show in research window and the research option disappears from the dock menu. I removed both files the mod places in the folder and replaced the original file.
« Last Edit: March 09, 2018, 09:51:27 PM by Burnthalo »