Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Putnam]

I will argue vehemently though that is not what 'safety' means.  Safety isn't about "protecting the programmer", it's an attribute of the code behavior.  Since it's demonstrably possible to have code that behaves safely without checks, and since it's demonstrable that code that is written with strong language checks can be "unsafe" in the real world, calling it 'safe' and 'unsafe' is actually more dangerous than some other word; I would call it 'managed' and 'unmanaged'.  If you argue instead that the checks make it easier to write safe code, I would agree - but the checks don't make safe code.

Memory safety is well-defined and is the term we're using here. Best to avoid equivocating.

[McTraveller]

On thinking about it - my biggest concern isn't that the unchecked portions are labeled as "unsafe¹" but that by inference people may think that code not so marked is safe, when it probably isn't - especially if they think it is because it's not in an unsafe section. Put another way: memory safety is a necessary condition of safety but it is nowhere close to sufficient.

Also remember, I work developing safety-related controls systems for a living, so I have a biased and strong opinion of what makes things "safe".

¹I do see that as a flag for "this code better be subject to much more stringent review and verification", not necessarily as an indicator that it's not actually safe.

Spoiler: P.S. (click to show/hide)

No, I don't work for Boeing or any of the big automakers. I cry inside every time I read about a controls failure because it points to a dysfunctional safety culture.

[methylatedspirit]

In an attempt to understand the interplay between C++ and assembler, I'm looking at C++ code and its generated assembler, and I need to ask. I'm using what's possibly the simplest non-trivial program in C++:

Code: [Select]

int main()
{
using namespace std;
for (int i = 1; i <= 100; i++)
{

}
}

Which just uses a for loop, incrementing a variable i starting from 1 until it's more than 100 (stopping at 100), if I've read it correctly. That generates (with flags: -m64 -Og -S):

Code: [Select]

.file "test.cpp"
.text
.def __main; .scl 2; .type 32; .endef
.globl main
.def main; .scl 2; .type 32; .endef
.seh_proc main
main:
.LFB0:
subq $40, %rsp
.seh_stackalloc 40
.seh_endprologue
call __main
movl $1, %eax
jmp .L2
.L3:
addl $1, %eax
.L2:
cmpl $100, %eax
jle .L3
movl $0, %eax
addq $40, %rsp
ret
.seh_endproc
.ident "GCC: (MinGW-W64 x86_64-posix-seh, built by Brecht Sanders) 11.1.0"

My questions are:
What's the stuff involving RSP doing? What's SUBQ doing to the RSP register?
Why is it just adding 1 to EAX instead of incrementing EAX with INC? I thought I wrote an increment in the source code.
What is it doing setting EAX to 0 after it's done? Is it memory protection, just in case the next process doesn't initialize EAX?

[McTraveller]

The manipulation of RSP is setting up the stack frame.  On x86, stacks grow "down", so subtracting 40 (hex or dec, doesn't really matter) is allocating space on the stack for local variables.

I haven't looked at x86 assembly in a long time, but either there isn't an inc eax, or it's using addl 1, eax because it's in debug mode, or maybe addl 1,eax is now an alias for inc eax and it will get the opcode for that anyway.  You can play around by looking at the -S output with -O perhaps (not -Og), or you can even change to pre-increment instead of post-increment

Code: [Select]

for (int i = 1; i <= 100; ++i)to see if that changes.

eax is set to 0 at the end because eax is used by default as the return value from functions. It's been forever since I've used C++ but I'm surprised you didn't get a warning for "main: reached end of non-void function without return statement".  Without a return statement, the default return value is 0.

[RoguelikeRazuka]

Hello, I came here to share my woes with a project I have been given on my current job recently, perhaps it would be desirable to hear some advice on how I should proceed in this situation in case someone has dealt with something similar (not sure if this really belongs in this thread).

So, I have been assigned a project. This project was not developed in our organization, rather it was handed over to us by the original contractor who also were not the one who originally developed this software and sold it to our client. The original contractor hired us as a subcontractor (according to my understanding, don't know the proper terminology) since apparently they lack the specialists who can handle this project.

The thing is, the project is not good. First of all, it's really complex. I lack the experience, the knowledge and, I fear, the brains necessary to work with this project. It takes hours for me to figure out how something works, since there is a lot of code, it's really complicated and hard to get into, it's complicated not only because there are some sophisticated algorithms or methods employed (although that complexity exists too), but in large part because the code operates on custom data/logic structures which refer to concepts and terminology which only the original developers themselves knew and understood well because they were the ones invented them in the first place, and I have no proper documentation to consult, all I can do is to inspect the code, but going like this it is very hard to get an general overview of the project so that I could navigate around it more freely and it was easier for me to get into the specific details, relate what the code does to what it should do, and understand why it was implemented in such a way.

The project had been in development for some 2-3 years I think, and apparently was partly rewritten several times, it was being worked on by a group of people whose level of competency and skill is far beyond my current capabilities, even more than that, judging by some of the FIXMEs and TODOs strewn around the code, the original developers themselves did not know or understand what to do in some situations, how to handle specific exceptions, fix specific bugs, more efficiently implement specific functionality, etc, and I am baffled as to how I am expected to solve all of these problems alone, if the necessity arises.

To reiterate, the project is quite complex and large (or so it appears to me at least, I haven't ever developed anything of a similar complexity and scale myself), it has tons of functionality of which operation I have no firm understanding. I scan the code with my eyes for hours and hours to no avail, I simply can't grasp its workings because of the aforementioned lack of experience, knowledge, and skill, but things are significantly complicated by the lack of proper documentation, and there is no one to guide me through the codebase, I do not know how I am supposed to fix stuff or even implement *new* things, because, as I said, there is a lot of code to handle by a single person, and there's no detailed and extensive documentation that I'm aware of, that would help me familiarise myself with it better. To further complicate things, it runs slowly on my machine, and when I try to run it in a debugger (I'm using the vscode Python debugger), to better analyze how things work,  the debugger just hangs for indefinite time when I try to look into a variable in the debug console (has someone encountered this bug in vscode? are there any workarounds?), so I have to use the print method to know what values the variables have etc, but it's awfully inconvenient, I have to add print statements for whatever variable I want to analyze, and then I have to restart the debugger and wait until control flow reaches that breakpoint again, which is wasting my time.

The project is quite ancient, works on Django 1.4.3 + Python 2.7, it's a legacy project, which I believe was abandoned by the authors themselves (though somehow they managed to sell it to the client 7-8 years back for a hefty sum of money), this terrifying mess runs excruciatingly slow and has awkward and confusing UI, the hapless users who have to use it on a daily basis according to the instructions do not know or understand (not that they are really interested to learn it anyway, as I take it) how it is organized, how it is supposed to function (specifically in terms of business logic, not actual implementation) and how to properly use it, for them it's all pure magic which just should work by itself, but the funny thing is that I, the developer, know very little about it myself, but despite that the users expect me to solve issues as if I were the very person who created this software the way it is (and made their lives miserable). I am required to process requests from users in the duration of several hours, depending on the severity of the problem, and if I fail to resolve requests in due time, the client is going to punish (I do not know the exact contract terms) my employer for non-fulfilment of the contract obligations. Another dismaying issue is that this software, despite being so sophisticated and having so many features, is in fact not very well suited to the specific use case for which it was it initially adopted, for, as I found out, this beast was conceived and developed more as a generic software platform for creating information systems than an application tailored for a specific field, it complicates things more than it aids the user, it's just too slow, confusing and awkward to use (it seems both users and developer suffer working with it), and to adapt it to the specific needs of the target user and generally make it more efficient and convenient I would have to come up with a generic solution, which requires deep understanding of the code, and I suspect large parts of it would have to be rewritten to change or add just a minor thing (if I want to do it the correct way, not resorting to ugly hacks, kludges and primitive duct-taping), and I've been struggling to wrap my head around it for quite some time already, maybe I'm just too dumb to do it, and I have no one to ask for help/advice/supervision since currently I'm the only Python/Django developer in the team (it wasn't the case some time ago, though, but things were not much different from now).

I did not know much about this project when I was first assigned it, I would likely not have accepted this offer if I knew beforehand. Working with this project is a nightmarish experience, it requires significant investments of time and effort, but gives very little in return, mostly headache.

So, do you think it's worth continuing struggling with this project, or it's better stop flogging the dead horse and accept defeat? Has any of you ever been in a similar situation?

[MorleyDev]

That time limit from clients is going to be the killer that makes it impossible. You need extra resources, either in time to process tickets or in extra people, a person can't handle both that and take control of a legacy code base at the same time.

An approach I've found useful for taking control of legacy code is after you identify the areas of code needed to be change, wrap that section in unit tests first and use them to document the current functionality. You may need to tweak the codebase first to make it testable (extract the area of code into functions/classes or introduce a DI system for mocking dependencies, IDE tools to do that automatically are useful for this when available). Only after it's covered in tests do you then change or add tests to capture the new/change functionality, and then you alter the code to make the tests pass again.

You could take a 'dual' approach. Fix the issue for the client in the headache inducing way first, and then use the time between tickets to 'revert' that and do it the 'proper' way. Or split between two people, one handling the immediate fires and one who works on the same ticket with the 'long term' fix that takes control of that area of code.

However, when dealing with single monolithic systems you can hit a point where it's easier to break them up and 'rewrite in parts' (avoid the big bang rewrite when possible). Often a monolithic service could be broken into a series of microservices, with a facade API that sits on top of the same database as the original project to 'make pretty' the API calls that under-the-hood could be ugly/painful operations.

All of these require business buy-in and resources to work though. Legacy code is code that people are afraid to change. Fear is because of risks. The business needs to understand that the risk is currently high, and steps are needed to lower it or it'll be a poison on the business.

[3man75]

Hi everyone. I'm trying to make an image pop up in Python after running some things in the console. I'm still a newbie.

I have tried googling to find the right module / library for this but it all seems kinda archaic. The modules available seem there for machine learning and image analysis. Where as all I want to do is simply open up an image.

I kept looking and now I find myself here: https://www.pythonforbeginners.com/files/reading-and-writing-files-in-python

Essentially, I can potentially use open() to open a file. But when I tried to copy and paste the name inside of the parenthesis I was given an error.

Code: [Select]

Arturia_Picture = Image.open("Artoria_Face_Shots_1.jpg", 'r')
    for line in Arturia_Picture:
        Arturia_Picture.write(line)

Apologies for the weak question. It seems you guys are dealing with higher level threats...And I can't afford to give out sandwiches for help atm due to covid.

[Bumber]

Code: [Select]

Arturia_Picture = Image.open("Artoria_Face_Shots_1.jpg", 'r')
    for line in Arturia_Picture:
        Arturia_Picture.write(line)


First thing that jumps out to me is that you're trying to write to a file you've just opened for reading. Searching for "python image.open" led me to this article, regarding the PIL module.

Would this work?

Code: [Select]

Arturia_Picture = Image.open("Artoria_Face_Shots_1.jpg", 'r')
Arturia_Picture.show()

[3man75]

Code: [Select]

Arturia_Picture = Image.open("Artoria_Face_Shots_1.jpg", 'r')
    for line in Arturia_Picture:
        Arturia_Picture.write(line)


First thing that jumps out to me is that you're trying to write to a file you've just opened for reading. Searching for "python image.open" led me to this article, regarding the PIL module.

Would this work?

Code: [Select]

Arturia_Picture = Image.open("Artoria_Face_Shots_1.jpg", 'r')
Arturia_Picture.show()

I'm going to try this in my code right now and let you know. Thank you!


EDIT: Okay so I uses some of the code as shown in the example that Bumber showed. But in addition I comented out that for loop as well as added the entire path.

I saw that as I added the entire path that Python gave me suggestions for what picture in my 'saved pictures' folder that I had. Picked the one I want and attempted to show it.

[Code/]

Arturia_Picture = Image.open("C:\Users\myName\Pictures\Saved Pictures\Artoria_Face_Shots_1.jpg", 'r')

    Arturia_Picture.show()

[Code\]

After running this code I get a truncation error. I'm going to do some research on what that is. A simple google search gave me this: "In numerical analysis and scientific computing, truncation error is the error caused by approximating a mathematical process. Let's take three examples so that the myths surrounding the definition of truncation error can be laid to rest."

Thank you Bumber for your help. I think I made progress on my little project here.

[lethosor]

Truncation in this context likely refers to the contents of the file being truncated, i.e. cut off, not numeric truncation. Some relevant pages:
https://en.wikipedia.org/wiki/Data_truncation
https://docs.python.org/3/library/functions.html#open

In general, it's very helpful to the rest of us if you copy and paste the error message you're getting (and in the case of Python, include the entire traceback). You can censor anything personal in it, of course, but having more detail generally helps other people identify the issue more quickly/reliably.

[Bumber]

Reposting this here, since it doesn't require DF-specific knowledge:

I reverse engineered the function for cutting announcement lines to an appropriate length based on grid size, and found some issues with it.
(See comments on: http://www.bay12games.com/dwarves/mantisbt/view.php?id=11813)

Equivalent code with testing is here:

Code: [Select]

#define crash_test false
#pragma warning(disable : 4996)
#include <cstring>
#include <string>
#include <iostream>
#include <vector>
using namespace std;

//function at 0x140657290, df_47_05_win 64-bit
void parseAnnounce(string input, vector<char*> &out_vector, size_t max_line_length)
{ //out_vector will contain cstrings cut to max_line_length, unless would cut word
if (input.length() == 0 || max_line_length == 0) //max_line_length of 0 results in subscript error
return;

bool ignore_space = false;
string current_line = "";
char *cstring_ptr;
size_t iter = 0;
do
{
if (ignore_space)
{
if (input[iter] == ' ')
continue;
ignore_space = false;
}

if (input[iter] == '&') //escape character
{
iter++; //ignore the '&'
if (iter >= input.length())
break;

if (input[iter] == 'r') //"&r" starts new line
{
if (current_line.length() != 0)
{ //push current_line as cstring to out_vector
cstring_ptr = new char[current_line.length() + 1];
strcpy(cstring_ptr, current_line.c_str());
out_vector.push_back(cstring_ptr);
current_line = "";
}
//push cstring " \0" to out_vector
cstring_ptr = new char[2];
strcpy(cstring_ptr, " ");
out_vector.push_back(cstring_ptr);
continue; //don't add 'r' to current_line
}
else if (input[iter] != '&')
{ //not "&&", don't add character to current_line
continue;
}
}

current_line += input[iter];
if (current_line.length() > max_line_length)
{
size_t iter_backup = iter;
while (input[--iter] != ' ' && iter > 0); //roll back to most recent space in input, ignoring char just added

if ((iter_backup - iter) != current_line.length()) //false when most recent space at end of previous line?
{
size_t post_space_index = current_line.length() - (iter_backup - iter); //index of char after space in current_line (usually)
if (post_space_index <= current_line.length()) //can be false with ampersands in input and low max_line_length, resulting in underflow
{
current_line.erase(post_space_index); //end string after the space (incorrect with enough ampersands in input)
}
else
{ //end up here if at least max_line_length+1 unbroken chars since start of line, post_space_index underflow
current_line.append(post_space_index, '\0'); //insufficient fix, just causes length error
}
//push current_line as cstring to out_vector
cstring_ptr = new char[current_line.length() + 1];
strcpy(cstring_ptr, current_line.c_str());
out_vector.push_back(cstring_ptr);
ignore_space = true;
current_line = "";
}
else
{
iter = iter_backup; //keep building this line until we get a space
}
}

} while (++iter < input.length());

if (current_line.length() != 0)
{ //push current_line as cstring to out_vector
cstring_ptr = new char[current_line.length() + 1];
strcpy(cstring_ptr, current_line.c_str());
out_vector.push_back(cstring_ptr);
}

return;
}

int main()
{
int grid_x = 80; //init setting: min 80, max 255 (probably)
//"The <profession> strikes at the <profession> but the shot is blocked!"
string text = "The &&aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa& strikes at the &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&& but the shot is blocked!"; //as appears in gamelog.txt
vector<char*> results;
cout << "\ngrid_x = " << grid_x << "\ntext = \"" << text;
cout << "\"\ncalling parseAnnounce(text, results, " << grid_x - 7 << ") ..." << endl;
parseAnnounce(text, results, grid_x-7); //the only calls to this function in DF use grid_x, grid_x-7, and grid_x-10

cout << "\nresults:" << endl;
for (auto i = results.begin(); i != results.end(); ++i)
{
cout << "\"" << *i << "\"" << endl;
}
//"The &aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
//"&&&&&&&&&&&&&&&&&&&but the shot is blocked!"

#if crash_test
cout << "\nClearing results vector ...\n" << endl;
results.clear();
grid_x = 16; //this is invalid, but lets us test with a shorter string
text = "&r123456789 and we've already crashed!";
cout << "grid_x = " << grid_x << "\ntext = \"" << text;
cout << "\"\ncalling parseAnnounce(text, results, " << grid_x - 7 << ") ..." << endl;
parseAnnounce(text, results, grid_x - 7); //results in string length error

cout << "\nresults:" << endl;
for (auto i = results.begin(); i != results.end(); ++i)
{
cout << "\"" << *i << "\"" << endl;
}
#endif
cout << "\nEnd of program." << endl;
return 0;
}

Is there an easy way to fix the issues with post_space_index, or would the whole function need to be rewritten?

[Telgin]

You're specifically looking to fix the problem where it cuts the line at the wrong index because of the escape sequences?  I'd need to step through this in a debugger to be sure, which I don't have time to do, but I'm guessing this is breaking because it's not doing the replacements on a temporary copy of the input and then splitting it.  When it tries to backtrack to a space and then resume where it left off, it's still seeing the unreplaced ampersands in the input and that's throwing things off.  There may be a way to fix it with just messing with post_space_index, but I'm not seeing any way that's not more complicated than rewriting this to do it in two steps.

[WealthyRadish]

Yeah, it shouldn't backtrack through the raw input string when seeking the start of a word to insert a line break, it should do so solely within the processed current_line std::string. It'll never work right without re-doing the escape sequence logic backwards, which is both unnecessary and really horrible.

[Bumber]

@Telgin, WealthyRadish
Thanks, I'll try that.

[McTraveller]

Sorry late to the party - couldn't this work by checking the rendered length of each word, then adding it to the line after confirming the length wouldn't exceed the limit?  This should work without having to backtrack.  I would do something like (pseudocode):

Code: [Select]

int length_so_far = 0;
while(not at end of input/format string)
{
    string next_word;
    int next_word_length, spaces_before_next_word;

    (spaces_before_next_word, next_word,next_word_length) = get_next_word(starting at = unprocessed part of input/format string);

    if( (length_so_far + spaces_before_next_word + next_word_length) < line_limit)
    {
       output_n_spaces(spaces_before_next_word);
       output_to_screen(next_word);
       length_so_far += spaces_before_next_word + next_word_length;
    }
    else
    {
        output_to_screen(newline);
        output_to_screen(next_word);
        length_so_far = next_word_length;
    }
}